As of early September, I’ve shifted to a new job in software development / 3D graphics with Mental Canvas. This comes after eight years as a data scientist / transport modeller at Metrolinx. I’m very excited to take on this new position, and I’m particularly keen to shift to a job where the product is working software instead of analysis and advice.
While the transportation modellers I’ve talked understand the close relationship between software and modelling, for others in transportation this seems like a drastic shift in direction. It’s not; the two jobs have quite a bit in common, and I’d like to explain the shared ground. To me, there are three basic stories here:
- Similar Data: both transportation and graphics have a lot of similarly structured data. Both jobs involve a lot of time organizing, structuring and automating the flow of data.
- Data Science & Software: a software person would use the term “data science” to describe transport modelling; it’s a hybrid of software and statistics, applied to the problems in the transportation field. Data Science and Software Development are closely related career paths.
- Software and Transportation are Converging: or perhaps more boldly, software is in the early stages of disrupting the transportation sector.
Continue reading From Data Science & Transportation to Software
I’ve long been a fan of data visualization, dating back to my days in the Imager lab at UBC, which had a research area in that subject. I’ve realized that my approach to “telling stories visually with data” includes a lot of knowledge that isn’t common in the transportation world, and I decided share what I know.
Drawing from the Internet, here’s a basic collection of content that gives a good introduction to how to communicate visually, using data. It’s even more compelling if you have my running commentary alongside… batteries not included.
- Telling Compelling Stories with Numbers
- Graphical Integrity
- Edward Tufte slides (selected slides; original here)
- The Human Visual System
- Tabular data
- Closing Thoughts & References
- Tufte, “Graphical Displays Should…” & Pantoliano
- Tufte, “Principles of Graphical Excellence”
- Kelleher & Wagener – Ten Guidelines
I haven’t covered one area in here: the basic principles of graphic design. But these are more widely known and can be learned in normal courses.
The agile process (or “scrum”) is very successful in the software world, but little known outside. My team has been doing scrums at a transit agency for 3-4 years, and I get asked about it regularly. I’ve assembled a few links that are useful for explaining it without being too jargony or software-specific.
I’ll talk more about:
- Visual Management
- Agile Product Development
- Scrum Process
Continue reading Scrums outside the Software world
A very quick update – I’ve finally updated my Toronto transit map to use more recent map information. As of March 2016, the maps now show data from roughly Dec. 2015 for all GTHA transit systems. (I hadn’t had time to update them since originally building the map in 2011.)
The main visible change is that the TTC map is now simpler and shows route frequency with the thickness of the lines. Unfortunately, I may have difficulty updating the TTC map going into the future – as of 2016, their map is now more “conceptual” and is not geographically accurate; I can’t readily warp it to sit on a geographically-accurate map.
Many years ago, I found an excellent resource for transit modelling: slides from a series of 2006-2009 workshops held by the US Federal Transit Administration (FTA) advising agencies applying for federal funding for rapid transit construction under the “New Starts” funding program. It’s very deeply buried on their website, and since then I’ve seen very few people reference this material, nor have I seen it assembled into a formal report.
So, for those interested – I’ve pulled together an easier-to-use table of contents to the three separate workshops, and tried to “deep link” into them to make it easier to browse and find the material. Enjoy!
UPDATE April 2016 – FTA has reorganized their website and the reports are no longer available there. I’ve mirrored everything here on my website.
Continue reading Federal Transit Administration (FTA) forecasting workshops
Modellers all learn about the different components of transit trip travel time, and the “perceived” weights that people put on them. It’s a useful insight into how transit works, and I find it’s a great exercise for testing how “useful” a new transit service is. The trouble is, after learning about weights, everyone wants to customize them – for their economic analysis, for one component of their model, etc. And analysis quickly gets inconsistent. Here’s why I think that’s often a bad idea – and why I think the weights used in transit assignment should be applied, unchanged, for all other parts of analysis. (And it’s not just me – the US Federal Transit Administration made this exact point in a 2006 discussion.)
Suppose that we have a four-stage model with different transit time weights: Continue reading Consistency in Time Weights
My employer recently hired Jarrett Walker to run his Transit Network Design course. I’m a long-standing fan of Jarrett’s work and regularly recommend his Human Transit book to colleagues.
I quite enjoyed the course, and it triggered thinking in three main areas for me:
- Pulses / Timed Transfer Hubs for low-density areas
- Completing the Grid using capital investment
- Designing for Coverage vs. Ridership on a single corridor: local-vs-rapid stop pattern
The course revolves around group “build a network” games, and those games will clearly play out differently depending on the attendance. Our course included several staff who work on bus scheduling, several who work on rapid transit planning, and many in areas further afield. That meant that each group had several experienced and numerically savvy staff, who could quickly get up and running with headways, frequencies and cost unit rates, without any tutorial. Continue reading Jarrett Walker’s "Transit Network Design" course
What makes a big organization work well?
Prior to my current job, my full-time working life was mostly in smaller companies. But working in a big company is a very different beast – making many teams and divisions move together is quite a task.
Some time ago, a friend pointed me to an interesting Wiki discussion on project management ideas. I followed a few threads there and stumbled on a fascinating book: Organizational Patterns of Agile Software Development by Coplien & Harrison.
It’s essentially a collection of ideas for improving organizations: building, healing, repair and growth. It’s written in a straightforward, browseable format: “if you have this problem, try this idea to fix it.” The book’s written for the software industry, but I feel most of the ideas are applicable in a wide range of fields.
Reading the book, what did I take away? Continue reading Organizational Patterns
A few years ago, I built a Google Maps app that combined the maps from several Toronto transit agencies all in one mashup map. I never got around to discussing the technical issues associated with that effort, and thought it might be worth writing up. This is an extra-technical post, covering the GIS / raster graphics / GDAL programming techniques I used to make the mashup work, for anyone else interested in trying a similar exercise. Continue reading Transit Map Mashup (Tech Talk)
I recently received an NGO request for the House of Commons members’ contact information in Comma Separated Value (CSV) format. I quickly found the helpful Represent website by Open North. This small group who has scraped government websites and created machine-readable versions of MP contact information, plus a minimal maps interface layered on top.
However, the Represent effort leans a little too “techno-elite.” The results are provided in the JSON format, a hyper-current, versatile and Web-buzzword-compliant format. But most NGOs I know are still struggling to figure out Excel, never mind anything more recent; and Excel can’t handle much more than CSV files (if even that – modern web-friendly text encodings like UTF8 don’t even work properly).
- Members of Legislative Assembly of Alberta, CSV format
- Members of Legislative Assembly of British Columbia, CSV format
- Members of Legislative Assembly of Manitoba, CSV format
- Members of Legislative Assembly of New Brunswick, CSV format
- Members of Newfoundland and Labrador House of Assembly, CSV format
- Members of Nova Scotia House of Assembly, CSV format
- Members of Legislative Assembly of Ontario, CSV format
- Members of Legislative Assembly of Prince Edward Island, CSV format
- Members of Assemblée nationale du Québec, CSV format
- Members of Legislative Assembly of Saskatchewan, CSV format