All posts by David

Represent Data

I recently received an NGO request for the House of Commons members’ contact information in Comma Separated Value (CSV) format. I quickly found the helpful Represent website by Open North. This small group who has scraped government websites and created machine-readable versions of MP contact information, plus a minimal maps interface layered on top.

However, the Represent effort leans a little too “techno-elite.” The results are provided in the JSON format, a hyper-current, versatile and Web-buzzword-compliant format. But most NGOs I know are still struggling to figure out Excel, never mind anything more recent; and Excel can’t handle much more than CSV files (if even that – modern web-friendly text encodings like UTF8 don’t even work properly).

So, I put together a quick bit of Javascript to convert the Represent JSON format to CSV format, ready for Excel to use. This should be “live” – pulling the latest version from Represent each time. Here are links to a few of the Represent datasets:

Federal

Provincial

Moving Cooler, part 2


Six weeks ago, I wrote a first piece on the Moving Cooler report. In that post, I walked through some of the report findings:

  1. Scope and approach
  2. What would happen if we deployed a sensible “bundle” of policies all at once?
  3. How do individual policies compare, within that bundle?

In this post, I’ll continue by looking at:

  1. What about at a “maximum” level?
  2. Conclusions

What does maximum deployment look like?

In the previous post, I looked at a few of the “big impact” policies. At the time, I focused on the “aggressive” deployment level. In this section, I want to switch gears to the “maximum” deployment level and dive a little deeper. Continue reading Moving Cooler, part 2

Moving Cooler, part 1

Three years ago, an excellent report advanced my understanding of which transportation/land use policies can really help to tackle climate change. From all appearances, that report has disappeared beneath the waves without a trace; I’ve met few policy advisors who have read it.

The report is Moving Cooler, written by consultants at Cambridge Systematics in 2009. It’s a non-academic technical piece with good math but poor messaging and graphics, and while there was some promotion during the study process, I’ve seen no follow-through. Together, this probably explains why it went unnoticed.

I’ll walk through the report findings in four stages:

  1. Scope and approach
  2. What would happen if we deployed a sensible “bundle” of policies all at once?
  3. How do individual policies compare, within that bundle?
  4. What about at a “maximum” level? (Discussed in part two.)

My own perspective in this post will be to understand two questions:

  • How quickly can we realistically hope to reduce emissions within the transportation sector?
  • Which policies offer the greatest potential at a reasonable cost? (This will necessarily ignore other considerations, such as equity or acceptability. To me, the first question is “does it work?” and only then is it worth asking “is it fair?” and “is it politically realistic?”

The report itself was never freely available, but executive summary material and appendices were online until quite recently. As the report website has recently disappeared, I’ve reposted a few of the freely available items for reference:

Scope and Approach

There are three key items to be aware of when interpreting the study:

  1. The study scope focuses only on policies in two of four areas:
    • included: travel activity changes by reducing the distance travelled or shifting to more carbon-efficient modes of travel.
    • included: vehicle and system operations by improving traffic flow.
    • not included: vehicle technology, such as electric vehicles
    • not included: fuel technology, such as cellulosic ethanol Continue reading Moving Cooler, part 1

Climate Change & Online Discussion

In the web world, the public conversation about climate change on mainstream discussion and media websites is quite distorted. The comments sections are filled with poorly informed opinions (on all sides of the debate), personal attacks and disinformation.  (It’s widely documented that the fossil fuel industries and other motivated groups are pouring significant amounts of money into propagating disinformation.)  There’s really no meaningful “conversation” happening on most sites, the comments usually shed little real light on the subject at hand, and I don’t think anyone changes their mind by reading the comments. See some examples here

I find it fascinating how real conversations are underway in some places on the web, though. Slashdot is a computer-science-geeks site I’ve been reading for 10+ years, with a very politically diverse spectrum of people on it. If you looked at most political articles there, the discussion is all over the place — left wing vs. right wing vs. libertarians vs. communists vs. anarchists vs. more libertarians.

But the forum is moderated, and well-moderated. And when it comes to climate change articles, the audience is technical, science-reading, and smart. Of the thousands of comments posted on each article, the readers vote the “best” ones up in a fairly effective manner using volunteer moderators and meta-moderators. Most heavily-debunked “denier” talking points are thoroughly rebutted and often voted right out of view. See for example

My recollection from reading Slashdot 5-6 years ago was the opposite — that the debate was much more vigorous, with denier talking points being moderated up equally with climate science talking points. But after scanning several 2004-07 posts, I don’t actually see that — Slashdot’s community and moderation system seems to have effectively rebutted the denier talking points for quite a while. See for example

It speaks to me about the value of websites with a real defined “community” and good moderation systems. It’s possible to learn a lot about the subject by reading Slashdot’s comment sections today, even though the site’s articles alone are minimal or poor. It’s also possible to take an interest in the subject without being repulsed by the constant name-calling, YELLING and bad writing in most newspapers’ comment sections…

This also makes me wonder if our political system can gauge the public opinion effectively. Is the “public’s opinion” the raw, unmoderated comments of a newspaper — or is it what would emerge if online newspapers with a fairly democratic / meritocratic moderation system like Slashdot’s?  (Sure, political offices rely primarily on polling; but they still learn the nuances of public opinion from letters to the editor and other such sources.)

Food for thought.

Advances in Population Synthesis, the journal article

I’ve finally published my M.A.Sc. thesis as a journal article, under the title Advances in Population Synthesis: fitting many attributes per agent and fitting to household and person margins simultaneously.

This article is the preferred citation going forward; I think it tells the story best:

  • A brief summary of the key contributions described in detail in my thesis
  • A better explanation of the U.S. context and the applicability of this work outside Canada. Statistics Canada goes to great lengths to protect Canadian privacy, and some of my work was motivated by the particular difficulties associated with Canadian census data.

My thesis is still a good source for anyone wanting greater detail, or anyone interested in a clear explanation of some of the Canadian data sources I used.

Continue reading Advances in Population Synthesis, the journal article

Greater Toronto Area transit map

Many years ago, when Google first released Google Maps and revolutionized online mapping from the stagnant MapQuest era, I put together a few quick demos showing the Vancouver and Toronto transit maps. I’ve made a few updates over the years since then, but not much more.  The Vancouver one is still quite popular – more popular than TransLink’s own map, to be honest – but other web gurus made better Toronto maps, such as the excellent one by Ian Stevens.

I’ve noticed that Google has revamped the mapping APIs and is preparing to eliminate version 2.0. The whole treatment of online mapping is changing rapidly, as the mobile market takes off. I was thinking of just scrapping the Toronto map since it’s not well-used – but then I thought a little further. What if I could make a proper map of the Greater Toronto Area?  Ian’s map doesn’t cover that – in fact, since there isn’t even a good print product covering the full area. Perhaps I could make something useful for the “regional traveller” using GO, and also help mobile users who have trouble with Ian’s site.

I set to work, borrowing liberally from others. It’s a patchwork by nature, since each agency has its own colour and line conventions, but hopefully still useful. When Ian made his map, taking a bitmap image and turning it into tiles was a bleeding-edge endeavour and required painstaking effort – but the tools have improved a lot. Even still, a bitmap this size (35,000 pixels square) takes some horsepower. I didn’t have the energy to do everything Ian did (like removing the background); his map will still probably work better for most TTC riders. I also couldn’t figure out what map projection Oakville Transit used, and couldn’t get it to line up nicely with the other data.

I’ll probably do a few more revisions on this in the next few months – an adjustable opacity slider would be nice, a legend for each operator, and higher zoom levels. But I thought I’d release a beta version and see if anyone likes it, and see how expensive the bandwidth is.

Version 3 of my map is now up (and version 2 is still around for anyone who wants it). New in this version:

  • Local transit operator maps
  • More mobile friendly: full-screen view by default, location-aware (uses GPS to detect your current location, if available)
  • May seem slower, unless you have a new browser, like Google Chrome or Firefox 4
  • Graphics updates: labels cleaner, interchange stations cleaner, labels always visible instead of showing on hover (for touchscreen users)
  • Search tries to find a transit station first, otherwise tries other non-transit locations
  • No legend… yet
  • Added “Get directions to here” link to each station
  • API version 3
  • Fixes: added Lincolnville station, fixed broken links

Canadian Government Scraps Long-Form Census

An appalling decision from the Canadian federal government today, reported by the Globe & Mail here: “Tories scrap mandatory long-form census”

The census is a vital data source for all sorts of transportation and land use planning.  A voluntary census is nearly useless, since the sample will suffer from voluntary response bias.  This will do nothing to reduce the number of analysts and bureaucrats – provincial governments will be forced to step in and collect the same data themselves, but this will inevitably result in the loss of province-to-province comparisons.

As for privacy, the alleged basis for this decision: Statistics Canada jumps through all sorts of hoops to ensure the privacy of respondents.  It would be difficult if not impossible to connect any of the published census data back to an individual.  Yes, the questions are detailed and probing; but the anonymization process used by Stats Can is tougher  than anywhere else in the world that I’ve seen.

Bryant and Road Rage

The judicial verdict is in on the sensational case of Michael Bryant. It sounds like a tragic case of a driver whose car’s stop/stall/roll action accidentally provoked an unstable bicycle courier, with tragic and ultimately fatal consequences. The driver appears to have behaved completely reasonably under the circumstances. The cyclist had a long history of aggressive confrontations and appears to have behaved in a threatening manner.

I take no issue with the facts of the case or the judgment. However, the incident has taken on large proportions in the media and the cycling community, and I find the official legal summary wanting in this regard.  The crisp, neutral judicial language gives the document the air of Truth and Justice, when in fact it only represents The Law.

The document answers the legal problem at hand, a judgment on dangerous driving, and therefore focuses entirely on what constitutes “reasonable driver behaviour” under the circumstances.  Lost in the context is what constitutes a reasonable cyclist’s reaction – and while the cyclist was not reasonable, an emotional reaction to having your rear wheel bumped is legitimate, and a subsequent furious reaction to being sent flying over the hood of the car is also fair.  Because the document is necessarily focused on the Law, all such points on the public debate of cyclist/driver perspectives and emotions are out of the picture.  Also lost in the discussion is any mention of the differences between car/car collisions from car/bike collisions – what may be a fender-bender in one context is an unnerving experience in the other, even if the vehicle doesn’t touch the cyclist’s skin, or if the vehicle is only going 13 km/h when it sends the cyclist flying over the hood.

The cycling community’s discontented reaction to the case stems from a desire for the driver world to “please understand our feelings!”  Unfortunately, this particular cyclist’s aggressiveness makes it an unlikely to elicit any soul-searching.  Note however the pattern: whether the cyclist is in the wrong (this case) or in the right (2008 story with a cyclist’s amputated leg), the cyclist will always be the one who gets injured or killed.

The morals: cyclists must hold their tempers, no matter the incident. And, there’s a profound lack of mutual understanding and respect still out there on the streets.

Winter cycling, Dutch style

Growing up in Toronto, I was a six-month cyclist and six-month pedestrian/transit rider. Since moving back a few years ago, I’ve been shifting to closer to ten months of cycling. I realized that I feel much better when I get that daily exercise and sunshine, and it’s considerably faster for getting around, chaining trips and running errands.

In the process, I’ve been trying to find the right bike for the job, and have just bought a pricy Dutch bike for the coming winter. My summer bike is out of the question; it’s a nice bike, and far too vulnerable to the winter salt, grit and filth.

Bicycle #1. I bought this road bike at the nadir of my student bank balance for $100. It’s as old as I am and a little too small. It handled reasonably well on the winter streets, the narrow tires were good at punching through the snow to find pavement, and the vintage handlebar-end shifters were easy to use with big mitts. The caliper brakes are the deal-breaker though: quite weak in wet conditions, and so tight around the wheel that I can’t have both fenders and knobbly tires. I rode it over the two winters of 2006-2008, when I lived in an apartment building. In that building, I could use the underground parking garage for cleaning and regular maintenance, and the bike thawed overnight in slightly-above-zero conditions. Rusting was a major problem: a new chain and rear cluster every spring, often cables as well, and a lot of surface oxidization wherever the paint had chipped off. Continue reading Winter cycling, Dutch style

Subcultures & cycling

The Copenhagenize blog has some interesting thoughts on cycling and subcultures: do the various cycling subcultures (racers, couriers, mountain bikers) in North America get in the way of making cycling appealing to normal citizens? The subcultures define themselves by gear or attitude – and I think this is quite offputting for normal people. It’s like a world where choosing to drive made everyone think you were a Formula One fan or a fix-your-own-car-guy. That said, the various European cycle chic blogs are perhaps guilty of pushing another subculture: beautiful people.