Metal Memory Debugging

At Mental Canvas, Dave Burke and I recently resolved a memory leak bug in our iOS app using the Metal graphics libraries. Memory management on Metal has had a few good WWDC talks but has surprisingly little chatter on the web, so I thought I’d share our process and findings.

Dave Burke and I recently resolved a memory leak bug in our iOS app using the Metal graphics libraries. Memory management on Metal has had a few good WWDC talks but has surprisingly little chatter on the web, so I thought I’d share our process and findings.

  1. The issue: a transient memory spike
  2. iOS / iPadOS memory limits
  3. Forming hypotheses
  4. Tools at hand
  5. Findings
  6. Conclusion

The issue: a transient memory spike

Our sketching app does a lot of rendering to intermediate, offscreen textures. In the process, we generate (and quickly discard) a lot of intermediate textures. A lot of that rendering happens in a continuous batch on a dedicated thread, without yielding execution back to the operating system.

A consequence of that behavior: our memory footprint grows rapidly until we hit the critical threshold — on a 3 GB iPad Air, about 1.8 GB — and then iOS summarily kills the app. From our app’s perspective, memory footprint is only growing gradually, as most of the memory usage is temporary intermediate rendertargets; they should be freed and available for reuse. But from the operating system’s perspective, it considers all of those temporaries to still be in use.

Which led us to the question: why? And how do we figure out which temporaries are leaking, and which Metal flag / call will allow us to return that data to the operating system?

Continue reading Metal Memory Debugging

Adventures in Vintage Emme

Imagine there are no variable names. Imagine working – in 2016 – with registers. Imagine one minute file load times. Imagine that all commands are just numbers. Imagine there’s no usable string processing.

Welcome to Emme 3. During the years that I worked in travel demand forecasting, this was the main tool available to me.

Emme was undoubtedly a trailblazing innovator when it first came out in 1982 and remained a power user’s dream through to the early 90s. But it clearly missed the Windows boat; the software seems to have stagnated until beginning a revival in the late 00s.

Emme 2’s graphical capabilities
Continue reading Adventures in Vintage Emme

Working with XAML Designer 2 (UwpSurface)

In September 2017, Microsoft released a rewritten XAML Designer program within Visual Studio. It’s only enabled for a tiny fraction of apps and got a very quiet launch, so almost no one knows about it. The new version runs as UwpSurface.exe instead of XDesProc.exe and is only enabled for UWP apps targetting Fall Creators Update SDK or newer.
For my app, the new Designer simply broke everything initially. Why? Presumably for technical reasons, it only works with {Binding} and not {x:Bind} – but this was not made at all clear in the launch announcements. UWP developers have been encouraged to use {x:Bind}: it’s compiled, type-safe and faster, and x:Bind functions are the only way to do multibinding. For my 64-bit app, I never got design data or design instances working under XAML Designer (xdesproc), so I relied entirely upon {x:Bind}‘s FallbackValue parameter to preview different modes of my UI – but {Binding} has no equivalent mechanism.
After a lot of tinkering, I’ve finally learned a few important things about the new XAML Designer, and got a usable workflow.

Top Takeaways

  • UwpSurface is much faster and stabler than XDesProc. The dialog above (“Click here to reload the designer”) is largely history.
  • It works for both 32-bit and 64-bit apps (x86 or x64), which is a big step forward
  • Only {Binding} is evaluated; {x:Bind} is completely ignored.
  • DesignInstances (a live ViewModel) can be attached via DataContext or d:DataContext, although the DesignInstance parameter doesn’t seem to work
  • ViewModel code executes, but I don’t see any indication that View code executes, despite discussion to the contrary in Microsoft’s launch blog post
  • For C++/CX apps: due to a bug, only single-class ViewModels (without any C++/CX base classes) work as of Visual Studio 15.8.x. They’re fine in C#.
    (Update Jan. 2019: Visual Studio 15.9 fixed this bug, but ViewModels that implement property change notifications still do not work.)
  • You can attach a debugger and see debug output from your ViewModel, or any exceptions. I haven’t been able to set breakpoints.
  • Update Jan. 2019: Visual Studio 15.9 added support for something I requested in mid-2018: the FallbackValue parameter was added to {Binding}, which makes that an equally viable way to work with XAML Designer. FallbackValue doesn’t work for {x:Bind} in the new Designer.

My strategy for a C++/CX app

  • Stick with {x:Bind} and its functions in most of my code
  • For the few properties that are central to a clean layout (usually about 2-5), use {Binding} and type converters
  • For those properties, build a duplicate “DesignTimeMock” ViewModel class – with no C++/CX base classes – and return the design-time property values there. Most properties can be safely omitted.
  • In the XAML code, define two different DataContexts like this:
    <UserControl.DataContext>
        <local:MyViewModel />
    </UserControl.DataContext>
    <d:UserControl.DataContext>
        <local:MyViewModel_DesignTimeMock/>
    </d:UserControl.DataContext>

    This attaches one ViewModel for runtime, and the mock for design time.

  • Update Jan. 2019: Visual Studio 15.9 now allows a different approach: instead of a DesignTimeMock, you can just have a single DataContext using the “real” view model, but instead change {x:Bind} to {Binding} in the few places where it matters, and then add a FallbackValue parameter to choose the desired value in XAML Designer. The biggest advantage of FallbackValue: you can edit it in place and see it immediately update, without recompiling and relinking the DesignTimeMock view model.

It’s not ideal. But… it’s better than nothing.

Closing Thoughts

Despite the difficulties, I do look forward to further improvements in the XAML Designer. The old version was dated and crash-prone, and a clean slate rewrite was the only reasonable path forward. It’s just going to take some time to reach feature parity with the old version, which will mean some teething pain for “guinea pig” developers like myself.

Asynchronous Best Practices in C++/CX (Part 2)

This is part two in a series of articles on asynchronous coding in C++/CX. See the introduction here.

  1. Prefer a task chain to nested tasks
  2. Be aware of thread scheduling rules
  3. Be aware of object and parameter lifetimes
  4. Consider the effect of OS suspend/resume
  5. Style: never put a try/catch block around a task chain.
  6. References

2. Be Aware of Thread Scheduling Rules

Asynchronous Best Practices in C++/CX (Part 1)

For me, the steepest learning curves with the Universal Windows Platform (UWP) was the use of asynchronous APIs and the various libraries for dealing with them. Any operation that may take more than 50ms is now asynchronous, and in many cases you can’t even call the synchronous equivalent from Win32 or C. This includes networking operations, file I/O, picker dialogs, hardware device enumeration and more. While these APIs are pretty natural when writing C# code, in C++/CX it tends to be a pretty ugly affair. After two years of use, I now have a few “best practices” to share.
C++/CX offers two different approaches for dealing with asynchronous operations:

  1. task continuations using the Parallel Patterns Library (PPL)
  2. coroutines (as of early 2016)

Personally, I vastly prefer coroutines; having co_await gives C++/CX a distinctly C# flavour, and the entire API starts to feel “natural.” However, at my current job we have not yet standardized on coroutines, and have a mix of both approaches instead. And to be fair – despite Microsoft’s assurances that they are “production ready”, I’ve personally hit a few coroutine bugs and they do occasionally completely break with compiler updates.
I’m going to write up my advice in a series of posts, as the examples can be pretty lengthy.

  1. Prefer a task chain to nested tasks
  2. Be aware of thread scheduling rules
  3. Stay aware of object and parameter lifetimes
  4. Consider the effect of OS suspend/resume
  5. Style: never put a try/catch block around a task chain.
  6. References

1. Prefer a Task Chain to Nested Tasks

When writing a series of API calls that need local variables, conditional logic, or loops, it’s tempting to write it as a nest of tasks. But a nest will:

This One Weird Grid Trick in XAML

I recently found a neat XAML user interface trick that I hadn’t seen in my usual resources. Suppose you have:

  • a grid-based responsive user interface that you want to grow/shrink to fit the window
  • suppose it has a fixed width, and each row has a “core” minimum height it needs.
  • then there’s some extra vertical space that you want to sprinkle around
  • you have some priorities – first give row #1 extra space, then row #2 any extra.

XAML makes it easy to do proportional space allocation – e.g., give row #1 two-thirds and row #2 one-third by giving them “2*” and “*” height respectively. But it doesn’t do priorities.
The trick: combine a massive star size with a MaxHeight. That looks like this:

<Grid>
  <Grid.RowDefinitions>
    <RowDefinition Height="1000*" MaxHeight="200" />
    <RowDefinition Height="*" />
  </Grid.RowDefinitions>
</Grid>

Essentially, row #1 gets “first claim” on any extra space, up to a limit of 200 pixels. Any extra space beyond 200 pixels falls over to row #2.

My First XAML Tips for the Universal Windows Platform

I’m a latecomer to Microsoft’s user interface technologies. I never used Windows Framework (WPF) on top of .net and I never used Silverlight on the web. The last year was my first taste of these tools through the XAML framework that is part of the “Universal Windows Platform” (UWP) – that is, the Windows 10 user interface layer (and Win8).
XAML has steadily evolved since the WPF days, and it took a little while to really understand the different major eras of the technology, especially since the UWP flavour of XAML strips out some of the older syntaxes in the name of efficiency on mobile platforms, better error checking at compile-time and code readability and ease-of-use. The technology’s old enough that much of the Google search hits and StackOverflow results are not applicable on the modern UWP platform.

My Tips

So what were a few of my first lessons when using XAML on UWP?
Continue reading My First XAML Tips for the Universal Windows Platform

From Data Science & Transportation to Software

mds
As of early September, I’ve shifted to a new job in software development / 3D graphics with Mental Canvas. This comes after eight years as a data scientist / transport modeller at Metrolinx. I’m very excited to take on this new position, and I’m particularly keen to shift to a job where the product is working software instead of analysis and advice.
While the transportation modellers I’ve talked understand the close relationship between software and modelling, for others in transportation this seems like a drastic shift in direction. It’s not; the two jobs have quite a bit in common, and I’d like to explain the shared ground. To me, there are three basic stories here:

  • Similar Data: both transportation and graphics have a lot of similarly structured data. Both jobs involve a lot of time organizing, structuring and automating the flow of data.
  • Data Science & Software: a software person would use the term “data science” to describe transport modelling; it’s a hybrid of software and statistics, applied to the problems in the transportation field. Data Science and Software Development are closely related career paths.
  • Software and Transportation are Converging: or perhaps more boldly, software is in the early stages of disrupting the transportation sector.

Continue reading From Data Science & Transportation to Software

Communicating Visually, Using Data

Picture1I’ve long been a fan of data visualization, dating back to my days in the Imager lab at UBC, which had a research area in that subject. I’ve realized that my approach to “telling stories visually with data” includes a lot of knowledge that isn’t common in the transportation world, and I decided share what I know.
Drawing from the Internet, here’s a basic collection of content that gives a good introduction to how to communicate visually, using data. It’s even more compelling if you have my running commentary alongside… batteries not included.

  1. Telling Compelling Stories with Numbers
  2. Graphical Integrity
    • Edward Tufte slides (selected slides; original here)
  3. The Human Visual System
  4. Colour
  5. Tabular data
  6. Maps
  7. Closing Thoughts & References
    • Examples
    • Tufte, “Graphical Displays Should…” & Pantoliano
    • Tufte, “Principles of Graphical Excellence”
    • Kelleher & Wagener – Ten Guidelines

I haven’t covered one area in here: the basic principles of graphic design. But these are more widely known and can be learned in normal courses.

Scrums outside the Software world

Scrumboard picture The agile process (or “scrum”) is very successful in the software world, but little known outside. My team has been doing scrums at a transit agency for 3-4 years, and I get asked about it regularly. I’ve assembled a few links that are useful for explaining it without being too jargony or software-specific.
I’ll talk more about:

  1. Visual Management
  2. Agile Product Development
  3. Scrum Process

Continue reading Scrums outside the Software world