Site menu:

Cyclocross For Roadies: How and Why to Get Started - Buy Now!

Site search

Categories

Clipping Large Shapefiles using OGR

My puzzle this morning was to clip a very large shapefile. I tried all of my usual methods (QGIS, Kosmo, etc.) and was having no success (crashing, hanging, taking forever, etc.) so I thought I would try out my old standby, OGR.

I had never used OGR to clip a file based on a polygon before, but had used it to clip to extents. The idea is the same, except that you specify the clip polygon layer rather than the extents in the -clipsrc option.  I headed to the ever-useful man page for ogr2ogr, and came up with the following, which worked perfectly!

ogr2ogr -clipsrc clipping_polygon.shp output.shp input.shp

It’s just that easy, and best of all it just works when all other methods fail!

Comments

Pingback from James Fee GIS Blog » Blog Archive » More Command Line GIS Goodness
Time April 1, 2011 at 12:51 pm

[…] are other ways to do this though. Darren Cope has a short blog post on using OGR for clipping GIS data files. Simple and sweet! It’s just that easy, and best of all it just works when all other methods […]

Comment from NoTime ToLose
Time April 11, 2011 at 7:54 pm

This…. More people should post their successes with the awesome, yet simple to use, tools. Some people are shocked that I don’t have to waste 10 minutes to open a full GIS Suite, discover data sources, click click repeat and …. result.
Thanks

Pingback from Using OGR | Ecostudies
Time April 24, 2011 at 3:15 am

[…] nice example is given in a post by Darren Cope about using OGR for clipping GIS data files. Using the ogr2ogr function, it only takes one line to clip a (potentially very large) vector […]

Comment from Gary
Time August 2, 2011 at 6:54 pm

Nice idea, but no chocolate fish.
A small test worked, but my big file (contours) didn’t. Clipping to x,y extents sort-of worked, but one of the y values was wrong…
So – any better suggestions?

Comment from darrencope
Time August 2, 2011 at 7:02 pm

Hi Gary. Any error message produced? What version of ogr are you using? What did the result look like?

Comment from Gary
Time August 28, 2011 at 6:13 pm

Oops. Looks like I might have made a basic error – the big file did not extend as far as I thought. It does work. Thanks, and sorry.

Comment from darrencope
Time August 29, 2011 at 7:26 am

Great! Glad it worked Gary!

Pingback from DARREN COPE » 2011 – The Year in Review
Time March 20, 2012 at 9:10 pm

[…] Clipping Large Shapefiles using OGR […]

Pingback from Public Data and Open Source GIS Tools Workflow | WriteSmith.com
Time April 1, 2012 at 3:33 pm

[…] slowed in our GIS desktop tool as a result. One approach that will help to deal with this is to clip the Census shape file to the extents of the City shape file, using ogr2ogr at the same time we project it. This will take a long time in ogr2ogr, but the one […]

Comment from Ruby
Time July 30, 2012 at 8:18 am

Cant believe how simple this was. I have spent hours with contour files trying to do what took minutes. Thanks very much!

Comment from darrencope
Time July 30, 2012 at 12:46 pm

Ruby; Great! Glad to help!

Comment from Andy
Time January 30, 2013 at 7:09 am

Hi – Stumbled across this whilst banging head against/trying to reduce a large and nasty polygon dataset. Got very quick, good results with FWtools for a smaller test but no joy with the beast: a shapefile is produced but it’s empty, and the following error messages come up: “TopologyException: side location conflict”. Any ideas? Clipping often a nightmare with large vector sets and finding a reliable solution would be very handy!! Thanks

Comment from darrencope
Time January 30, 2013 at 8:14 am

Hi Andy,

Sounds like your shapefile is corrupt. Two things I would try: one, run it through ogr2ogr and specify -skipfailures. This will drop the invalid features (not ideal, but may at least allow you to find where they are.) The second option would be to import into GRASS, which will build topology on the file and hopefully correct any issues. You could go from there back out to shapefile if you wanted. Hopefully one of those options works for you!

Comment from Andy
Time February 5, 2013 at 9:58 am

Hi Darren – it certainly was corrupt, and has taken a few revisits to get sorted… But eventually ogr2ogr worked, when others fell down too. Thanks for the tips!

Comment from darrencope
Time February 5, 2013 at 10:03 am

Andy: Great! Glad you finally got it sorted! Did you end up using GRASS to do the cleanup? Or some other method?

Comment from JeffT
Time June 3, 2014 at 11:38 am

When clipping against large shapefiles it can be more efficient to do it in two steps. First clip to the extent using -spat option. Second clip to the shapefile as shown in the command above.

Comment from darrencope
Time June 4, 2014 at 6:50 am

Thanks for the comment Jeff–good point!

Comment from Emily
Time March 22, 2015 at 2:38 pm

Hi Darren, I’m new to working with shapefiles in R. Do you think I could clip a point shapefile consisting of ~200,000 points to a polygon shapefile consisting of ~30 polygons (they represent certain districts)–in the way you have suggested here? Thank you!!!!

Comment from darrencope
Time March 26, 2015 at 7:07 am

Hi Emily; I don’t see why it wouldn’t work. I’m not familiar with R myself, but if you can call ogr, I bet it will work!

Pingback from A map with the number of mammal species based on IUCN redlist range maps | Ecostudies
Time April 3, 2015 at 12:07 pm

[…] The maps are available as shapefiles. Not as one layer per species, but one (very) large shapefile that contains all the distribution maps of that group. If you don’t need it for the whole world, the first step you probably want to do is to clip the layer to your area of interest (there are various tools available in QGIS or you can do this from the command line using ogr2ogr. […]

Comment from tristan
Time June 16, 2015 at 12:33 pm

Hi Darren,
I am currently working on mangroves over sundurbans in Louisiana, and I have a shapefile – geometry type Polygon – for those mangroves (+100 000), and I need to clip this shapefile according to another shapefile (only one polygon in this one), representing my region of interest.
I need to do that to speed up my further processes, first being gdalwarp over a raster according to the clipped mangroves’ shapefile !

But it seems very slow !!! Already ~1,5hour…

Thank you!

Comment from darrencope
Time June 16, 2015 at 12:35 pm

Tristan; it should work–let it run for a while longer and see what happens. Is the process still responding? If so, you should be all set!

Comment from tristan
Time June 16, 2015 at 12:47 pm

Thank you for your quick answer !
Yeah it is responding, but I am surprised that it is so long.
When I am running it through QGIS with the Vector > Geoprocessing Tools > Clip tool, it takes ~30s…
Do you have any idea if the ogr2ogr command is used in QGIS as several gdal commands are used for rasters’ processings ?

Comment from darrencope
Time June 17, 2015 at 6:45 am

Hi Tristan; I’m not sure what you mean. If it works in the Geoprocessing tools, go with it! As far as I know, the QGIS Geoprocessing tools are not using ogr, so they will behave differently. Did ogr end up workin in the end? If not, is there a chance you could send me the data to look at?

Comment from tristan
Time June 17, 2015 at 11:50 am

I was told than ogr2ogr is the fastest way to work shapefiles…
Eventually, after 3,7hours I have the result I was expecting, but it is way too long… I might have something like 30 of this type of computations to execute.
I don’t understand why it is so slow…

Comment from darrencope
Time June 18, 2015 at 11:21 am

In general, OGR is faster for most things. I’m surprised as well. Do you have a spatial index built that OGR can use?

Write a comment





Bookshelf 2.0 developed by revood.com

More in GIS (20 of 35 articles)


I spent my morning today playing with some of the new data formats in OGR 1.8 (see the release notes ...