I’m not even sure how best to summarize 2011; once again it was a year of ups and downs, both in life in general, and in my cycling. However, like 2010, I’m very fortunate that the ups outnumber the downs by a significant degree! If only I could figure out how to eliminate those downs all together…

Cycling

In 2011, I got out on the bike 150 times, for a total of 6882.96km. This is down 964.01km from 2010, which seems hard to believe because it doesn’t feel like I did significantly less riding! Total ride time clocked in at 239.74hrs, which is (obviously) 10 full days  (or, put another way, 2.7% of the year.) Some quick math shows that my average speed (for all rides, on all bikes) was 28.71km/hr. These numbers all exclude the 38.62hrs I put in on the spin bike, which would bring my total to 3.2% of the year ;)

The first outdoor ride of the year was February 5, and the last was on December 11. My biggest month was July when I rode 1424.46km, while the smallest was January, with zero on-road kms :(

As for bikes, I put 5380.94km on the Colnago (78% of my total), 1047.83km on the ‘cross bike (15%), 345.84km on the TT bike (5%) and a mere 22.12km on the Mountain bike (0.3%). A couple of others thrown in there make up the last few percentage (test ride on the R3, etc.)

Early season highlights included my best ever result at Paris-Ancaster, coming 75th overall (in a field of 1188) – 26th of 284 in the Men 30-39 category. Total time was 2h 02′ 40″, which was much better than the 2h 05′ 34″ from 2010! I hope to have another improved result this year, but perhaps shouldn’t get too confident! It’s hard to continue to knock time off year after year!

An early season minor crash slowed me down a bit in May, but fortunately not much damage was done! Summer consisted of lots of good fast rides, with every Monday for a month and a half faster than the previous! A good hard Rideau Lakes tour was thrown in for my longest ride of the year – 192.38km.

I was once again unsuccessful in my attempt at the 40km Project, coming 22 seconds from my goal. Frustrating to say the least! As is clear from some of the numbers above, I need to spend more time on the TT bike – 446km/5% just isn’t enough to be fully comfortable yet! This will once again be the primary riding goal for 2012.

Without a doubt, the overall riding highlight of the year was Tour de Brew; Vermont hills are amazing and I can’t wait to go back for 2012! Planning is already underway for an even bigger and better trip!

Fall ‘cross season was about the same as always; nothing special! By the end, I was once again lacking motivation, and entirely burnt out! If only ‘cross season was in the summer ;) On the plus side, increased focus on core strengthening and a slightly tweaked bike fit resulted in an almost entirely pain free season–the first since I’ve started having back issues! Goal for 2012 is to continue doing strength and stretch work all year rather than just when problems become noticeable! (Yes, I realize I said that last year…)

Paddling

Well, once again, I only got out for one quick trip this year; to Puzzle Lake Provincial Park with Matt. The string of twice a year trips seems to have ended :( I hope to rectify that this year, and get in at least a couple of short trips. Ideally at least one of them would even be a long trip!

 Blog

The blog has been up and down about as much as everything else.  I go through phases where I really want to write, and then phases where it’s the last thing I want to do! According to Google Analytics, this site saw 10,131 visits and 16,454 pageviews in 2011. The 10 most popular posts were:

• Sigma Rox 9.0 – Part II
• Clipping Large Shapefiles using OGR
• Sigma Rox 9.1 – Initial Review and Thoughts
• Merge A Directory of Shapefiles Using OGR
• Google Maps + Firefox Smart Keywords
• Map Books in QGIS
• QGIS Topological Editing
• QGIS Diagrams – Pie Charts for Symbols!
• Sigma Rox 9.0 Follow-up Review – One Year In
• QGIS Trace Edit Tool

Interestingly, my Sigma Rox posts continue to be very popular, and the rest of my traffic tends to be more GIS related.

My eBook continues to sell, slowly. The big news this year is that, as well as being available from the Road Bike Rider eBookstore, you can now buy your copy directly from this site for a much lower price–only $1.99!

Overall, another fun year with some great rides and good times! Here’s to an even better 2012!

StatsCan Data Release Dates

Hi Everyone!

Just a quick reminder that the Statistics Canada Census Geography data should be released today for free! I don’t see it there yet, but I’ll keep checking!
EDIT (9:05am): It looks like the data is now available!

I’m happy to say that the 0.3.9 version of QGIS’ “Table Manager” Plugin by Borys Jurgiel now allows you to change a shapefile schema without having to save the output to a new file!

You can install the plugin via the Plugin Installer – look for “Table Manager.” Once it’s installed, find it under “Plugins > Table > Table Manager.” It’ll operate on the feature you have selected in the legend, and open up a simple window that looks like this:

Table Manager

You can then reorder columns, rename columns, insert columns, clone (copy) columns, change data types, and delete columns! To be fair, you could do all of this before. The difference is that now you can just hit “Save” down at the bottom, and carry on! Before, you would have to “Save As,” browse to a folder, and create a whole new file. Then add this new file to your map, theme it appropriately, and remove the old layer. Annoying. Now, it’s a super simple one-step process!

A nice touch is that the old .dbf file is archived in case of a mishap. The archive is stored alongside the rest of the shapefile, but has the extension .dbf~

Thanks Borys!

Now, if only this was built in to the “Properties > Fields” tab! ;)

Control Rendering Order

You’ll see that it is split into two panes, with one called “Layers” and one called “Layer order.”  Each now has a checkbox to “Control rendering order”

Essentially, the new functionality allows you to decouple the way layers are organized in the legend (“Layers” pane) from the order they are rendered (drawn) on the canvas (controlled via the “Layer order” pane.) This is really nice if you have complex maps with many layers and groups in the legend!

It still seems a bit buggy, but considering it just came out in last night’s nightly build, I’m not surprised! I’m looking forward to seeing this mature!

Installation:

EasyPrint is written by Stefan Ziegler, who we all owe a big thanks! You can install the plugin from the “Fetch Python Plugins” menu. If you don’t see it there, you may need to add the CatAIS Repository. To do so, go to “Plugins > Fetch Python Plugins” and click the “Repositories” tab. At the bottom, click “Add 3rd party repositories,” then “OK.” Then switch back to the “Plugins” tab, and type”EasyPrint” in the search. You should have one result, which you should select before clicking “Install plugin.”

EasyPrint in the Plugin Installer

Once the plugin is installed, you can enable it under the “Plugins > Manage Plugins” menu if it is not already enabled. If the install was successful, you can access “EasyPrint” in your plugins menu, or via the toolbar icon that looks like this:

EasyPrint Icon

If you already have the plugin installed, you may want to see if there’s an updated version–the new version seems to increase stability quite a bit!

Functionality:

EasyPrint has several functions. When you click the icon, you’ll get a three-tabbed interface that looks like this:

EasyPrint GUI

The first tab, “SimpleMap” allows you to create a one-page map, very similar to the standard QGIS Composer window. It uses layouts that come with EasyPrint or one that you can customize (more about that later.) In fact, this option even opens the EasyPrint layout in the composer window. To me, this tab seems unneccessary, because this process is more difficult than using the standard composer. Thus, I don’t see a real need for the SimpleMap option at all, except that it lets you quickly turn on or off things like the grid, copyright, etc. via checkbox rather than adding or removing them from a layout. However, it’s important to note that the next two tabs will use some of the settings from this first tab, so make sure you note them and set them appropriately.

Where EasyPrint really shines is in the ability to automate the creation of multi-page map books, rather than just simple maps. The second tab, “Mapbook by grid” allows you to print one page per ’tile’ based on a grid it will create. Make sure to set the scale you want your maps to be created at by choosing it on the first tab. EasyPrint will use the extent of the layer you choose under “Map layer” when creating the grid. I found this a bit confusing, and it took me a while to figure it out. You can also tell EasyPrint if you want to print all grid tiles (Regular Grid,) or only tiles that actually contain features on your chosen map layer (Regular Grid w/o empty grids option). If you select the ‘w/o empty grid’ option, EasyPrint will check if there is a feature present in “Map Layer” you have selected, and if so, create a page for that map tile. If not, it will skip it and move to the next tile. Once you’ve set these options appropriately, hit “Create.” You’ll see a new layer called “Mapbook Grid” appear in the layer legend as EasyPrint creates the grid. Note that this is where EasyPrint often crashes for me, throwing a fun Python Error. Save your work before hitting ‘Create’ and just try try it again. It almost always works the second time! Once the grid is created, it’s as simple as selecting any of the optional parameters, and telling EasyPrint where to export your file(s). Then hit “OK” and wait for your .pdf(s) to be created!

Here’s a quick example of a mapbook created using the “Regular Grid” function. Note that tile 2.1 is empty, and would have been skipped if I had selected “Regular Grid w/o empty grid”)

Page 1

Page 2

Page 3

Page 4

Page 5

Page 6

Page 7

The third tab, “Mapbook by feature” is the one that drew me to the tool in the first place. It works similarly to the “Mapbook by Grid” tab, but rather than creating tiles at a fixed scale in a grid pattern, it creates a sheet for each individual feature in a layer. The scale will vary based on the size of the feature, and you can set the amount of ‘buffer’ space around each feature to give some context to each map. You can also choose attributes from the table to use as titles and subtitles, and these will change on every page. Sweet!

Here’s an example of the same dataset shown above, but using the “Mapbook by feature” tab this time. Note that it only creates a page for each of the three features, and the scale varies depending on the feature size. Also note the titles, which change on each page based on data in the attribute table. In my case, they are the very useful (“One,” “Two,” and “Three”) Pretty slick!

Page 1

Page 2

Page 3

Page 4

Layouts:

By default, EasyPrint comes with three layouts. Unfortunately, they are not very useful for anyone except the person who created them! Thus, you’ll very likely want to create and use your own. Because this post is getting long, and because I’m still playing with the layout customization, I will leave that topic for a new post! It’s (unfortunately) not a super user friendly process, so I’ll try to break it down in the next post. Stay tuned! (If you want to get started earlier than that, take a poke at layouts.xml in your \user\.qgis\python\plugins\easyprint\layouts\ folder!)

Conclusion:

This is a fantastic plugin! Despite crashing occasionally (save your Project often!) it does a great job when it works.

My wish list for this one is pretty short right now. Basically, I’d love to see this map book creation functionality built into the core Composer window in QGIS, or at the very least have the EasyPrint plugin extended to include support for templates created via the composer. Some modifications to ‘intuitiveness’ (like not using settings from the first tab on subsequent tabs) would be nice, but not critical. It all works once you get the hang of it!

As an example, I created a random sample dataset with some fictitious data. It contained three points, and the attribute table looked like this:

Attribute Table

To create a diagram symbol, go to the properties for the layer in question, and click the “Diagrams” tab at the top, then check off “Display Diagrams” and set Diagram type to “Pie Chart.” You can also use “Text Diagram” but that’s another post!

You can set either a fixed size, or scale the size of the charts to an attribute value range. This is useful if you are wanting to use the size of the symbol to indicate a value, and the ‘slices’ to indicate percentages. In my case, I set the size to ’23′ and set it to scale between 0 and 30mm. This is arbitrary in my case, but you can play with it using your own data to see what works best for you.

At the bottom under “Attributes,” select each attribute you want to form a ‘slice’ of the pie, and click the ‘+’ on the right. Then select the next value, and click ‘+.’  Repeat until you are finished adding all values. In my case, I added attributes A, B, C, and D. I didn’t want to use the ID, so I left it out. You can set individual colours for each ‘slice’ here by double clicking the colour in the table. Here’s what my settings looked like:

Pie Chart Settings

and here’s here what the final product looks like:

Final Result

This is already a great tool, and with a few more enhancements, it will be a fantastic tool! What enhancements you wonder? A couple that immediately come to mind are the ability to label ‘slices’ with either their attribute value, or their percentage, and the ability to remove all borders (in the above example, ‘Pen Width’ is set to 0, but it still shows a border. Weird.)

What features would you like to see?

In a previous post (QGIS Trace Edit Tool) I discussed how to use QGIS’ trace edit plugin to create a polygon adjacent to an existing polygon while ensuring that it is topologically correct. With a bit more experimentation, I’ve found a better and easier way to perform the same task, using QGIS’ built-in topological editing functions. It’s easy and super convenient!

The key is that you’ll need to go to Settings > Snapping Options and enable the “Avoid Int.”  checkbox. (The mouseover reads “Avoid intersections of new polygons” and the checkbox doesn’t appear for lines or points, since it doesn’t apply.)

Enable "Avoid Intersections"

You can then digitize a polygon beside existing polygons, and have the new feature ‘clipped’ to the boundary. This is faster than using the trace edit plugin like I mentioned before, and it also ensures that no vertices are missed! Fantastic! How about we walk through an example? Pretend we had these existing features:

Existing Features

Now, let us pretend we want to draw a polygon that fills in the hole in the middle. We could simply set our snapping options, zoom way in, and try to snap to every vertex using the standard edit tool. Or, we could use the trace edit tool, and ‘trace’ around the boundary while holding the Ctrl key. OR, we could set the “Avoid intersection” option as shown above, and just quickly sketch in the boundary like so:

New feature boundary

The red feature above is what I drew in. No need to go carefully, or try to snap to anything. Just roughly sketch in the area I want. Topology rules take care of the rest, clipping the feature to the existing boundary when I finish my sketch. The final product looks like this:

Finished Product

Voila! It’s just that easy! I quickly (not precisely at all) drew in the rough shape required, and ended up with a perfectly topologically correct feature that follows the existing boundary exactly. No need to snap or trace anything! Sweet!

However, as with most open source projects, the documentation is often lacking. I found this to be the case when I wanted to use the “Trace Edit” tool. It’s a plugin, and installed using the Plugin Fetch & Manage functionality of QGIS. Just search for “TraceDigitize” in the repositories, and click “Install Plugin.” I assumed that the tool would allow me to make a new feature that ‘snaps’ to an existing feature by tracing along the edge of the existing feature, without having to find and click every vertex. It turns out that this is exactly what the tool does…however I just couldn’t seem to get the tool to work! A Google search turned up nothing except for some developer chats about bugs and QGIS crashing while using the tool. Great!

After some frustration and experimentation, I got the tool to work. So, what’s the secret? First, you must have the “Snapping Options” set (Under Settings > Snapping Options) as the tool requires these settings to know what to snap to. Secondly, you must hold the Ctrl key while digitizing. No, this is not documented anywhere. Until now. If you don’t hold Ctrl, the tool works just like the normal digitizing tool.

Trace Edit Tool

One thing to note is that if your snapping tolerance is too small, or you go too fast as you trace, you will still end up with gaps as in the example above. Try tweaking (increasing) the snapping tolerance, or slow down a bit, and you should be fine!

Enjoy!

"Cannot open the mapset. Mapset is already in use."

Depending on which version of GRASS you’re running, or your OS, the dialog box may look different (this one shown is from a GRASS session running inside QGIS; coincidentally my fvourite way to run GRASS….) but the message will be the same, reading “Cannot open the mapset. Mapset is already in use.” So, what does it mean? It means that for some reason (likely due to a system crash, improper exiting of the program, or a hung module…) your previous GRASS session was not properly closed.  As GRASS was originally designed for a multi-user environment, it protects each individual mapset from being edited by more than one person by locking it when it is opened. It unlocks the mapset when it is closed; unless something goes wrong as I mentioned above.

When this does happen, you’ll be unable to use the mapset, even though you know no one else is using it. So what do you do? It’s easy really!

Browse to the mapset in your OS filesystem. There should be a file called ‘.gislock’ there. Delete it. You should then be able to open your mapset normally!

Note that you are just doing what GRASS does normally when it works properly; .gislock is the file created when you open a mapset, and when a mapset is closed properly, it is deleted. You deleting it manually won’t change or break anything. Just be careful not to touch any of the other files in the mapset!

There you go! It’s just that easy!

We’ve all had a Shapefile with column names that make no sense (10 character limit anyone?) or with fields that were manually created by some geoprocessing tool and given a random name upon output. So how do you rename these columns? QGIS is getting close with its new ‘Fields’ options under the layer “Properties,” but can’t yet do the job. The “Table Manager” plugin will do the job fairly easily, but is not a built in function. Kosmo will do the task, but is often very slow and will choke on a large file. And please don’t go opening the .dbf and editing names directly–that’s just asking for trouble! (and Excel won’t even save back to a .dbf, so you’ll have to use LibreOffice!)  So, what’s the failproof solution? Once again, we turn to the amazing OGR. Using ogr2ogr we can slide in an SQL statement that does the trick nicely, and as with almost all OGR tasks, it’s quick, stable and effective. Here’s how simple it is:

ogr2ogr outdataset indataset -sql "SELECT src_field1 AS dst_field1, src_field2 AS dst_field2 FROM sourcelayer"

Of course you can rename any/all of the fields using this method all in one step! The other great thing is that you can enter virtually any SQL statement, so you can do other tasks at the same time as the rename if you need to. For example you could pick out select fields, rename them, and also enter a WHERE clause to subset your dataset at the same time. There are so many options!

Final word: Don’t be afraid of the command line! It’s fast, stable and powerful!