OS OpenSpace [logo]

OS OpenSpace Forum » OS OpenSpace » API Development Help

Thread: Displaying large numbers of markers

Reply to this Thread Reply to this Thread Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 9 - Pages: 1 - Last Post: 05-Jan-2013 20:13 by: sacapuce Threads: [ Previous | Next ]
sacapuce

Posts: 15
Registered: 19/02/10
Displaying large numbers of markers
Posted: 26-Aug-2012 21:40
  Click to reply to this thread Reply

Hi,
I am trying to build a map of more than 20,000 points (markers) for survey sites with some data and a hyperlink to a photo per site.

I created the markers using a largish file (5 Meg) and a loop and it worked well on Google Chrome and to some extent on Firefox (although it takes some 30 seconds for all markers to load).

In Internet Explorer 9, the map loads but as soon as I zoom in or pan, I get all sorts of problems and the markers info window does not display well, if at all.

My guess is that the problem comes from the number of markers I created.

Any suggestions as to what I can do?

Shall I split the markers file into several files and load them as separate layers that can be made visible or invisible? Is that feasible (can I create a layer of markers?) and how can it be done (any links to code or examples)?

Are there ways of dinamically loading markers' layers?

Many thanks
Marc

Message was edited by: sacapuce

prmaps

Posts: 170
Registered: 28/10/08
Re: Displaying large numbers of markers
Posted: 27-Aug-2012 11:18   in response to: sacapuce
  Click to reply to this thread Reply

> In Internet Explorer 9, the map loads but as soon as
> I zoom in or pan, I get all sorts of problems and the
> markers info window does not display well, if at
> all.

in IE9, make sure you're in standards mode and, if using vectors, using SVG (which is faster to display) and not VML

> Shall I split the markers file into several files and
> load them as separate layers that can be made visible
> or invisible? Is that feasible and how can it be
> done (any links to code or examples)?

yes. it's highly unlikely that all users want to look at all 20,000 markers, so it's unfriendly to force them to load all of them whether they want to look at them or not. First off, use vector features, not markers. Either cluster them, so you see more as you zoom in; only display them above a certain zoom level and use a BBOX strategy to read in those for the current viewport; and/or split them into categories in different layers. There's an example of the last in my OL tutorial http://www.peterrobins.co.uk/it/ollatency.html and http://www.peterrobins.co.uk/it/olstyle.html and see the OL examples for clustering.

sacapuce

Posts: 15
Registered: 19/02/10
Re: Displaying large numbers of markers
Posted: 16-Sep-2012 13:23   in response to: prmaps
  Click to reply to this thread Reply

Many thanks for your advice. I will look into the clustering and BBOX strategy option as I cannot really split the dataset in a meaningful way.
I have had a look at your website and tutorials and I would recommend them to anybody new to the world of OpenLayers. Many thanks for putting that online.
Cheers
Marc

sacapuce

Posts: 15
Registered: 19/02/10
Re: Displaying large numbers of markers
Posted: 17-Sep-2012 19:57   in response to: prmaps
  Click to reply to this thread Reply

Hi,
I managed to turn my 24000 points into a GeoJSON vector layer (as I got loads of problems with kml) and load it on the OSopenspace map using a BBOX strategy and a zoom threshold as you suggested.

It works fine apart from IE9. You mentioned SVG and VML. How do I use SVG? Do I need to build it as part of the code and could you direct me to any example?

Thanks
marc

prmaps

Posts: 170
Registered: 28/10/08
Re: Displaying large numbers of markers
Posted: 18-Sep-2012 16:38   in response to: sacapuce
  Click to reply to this thread Reply

> It works fine apart from IE9. You mentioned SVG and
> VML. How do I use SVG? Do I need to build it as
> part of the code and could you direct me to any
> example?

If you're using the OpenSpace api, that includes both SVG and VML. Version 9 was the first version of IE to include SVG rendering and, as long as you're not specifying a compatibility mode http://msdn.microsoft.com/en-us/library/cc288325(v=vs.85).aspx (or are using edge) OL vectors should use that in preference to VML. You can check in the javascript console; the renderer object in your vector layer should be an instance of OpenLayers.Renderer.SVG.
I've not heard of problems with vector rendering in IE9, but I've never tried with large numbers of vectors, and I don't know whether there are still issues. OpenSpace is using an old version of OL, and there may be problems there you wouldn't get if using the latest OL.

If all else fails, advise your users not to use IE :-)

sacapuce

Posts: 15
Registered: 19/02/10
Re: Displaying large numbers of markers
Posted: 19-Sep-2012 16:01   in response to: prmaps
  Click to reply to this thread Reply

Thanks for your response. I had a look and compared Chrome and IE9 consoles and found that in the map in IE9, the vector layer appeared with the following SVG tagged lines:

<svg id="OpenLayers.Layer.Vector_58_svgRoot" viewBox="0 0 900 700" width="900" height="700">
<g id="OpenLayers.Layer.Vector_58_root" style="visibility: visible;" transform="">
<g id="OpenLayers.Layer.Vector_58_vroot"/>
<g id="OpenLayers.Layer.Vector_58_troot"/>

The only difference between the two is that under Chrome first "id="OpenLayers.Layer.Vector_58_troot" line, the points visible in the map were listed whereas nothing appeared in IE9, hence the absence of data on screen.

I checked the cached area on IE9 and found the JSON file (as a http file) containing the 24000 points along with a reference to the bbox boundaries so I am a bit puzzled as IE9 seems to download the file, use the SVG but somehow not display the points.

As for the compatibility mode, it is set to html5 so it should be ok (see below)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">

Any idea why it is not displaying?
Many thanks for your help

prmaps

Posts: 170
Registered: 28/10/08
Re: Displaying large numbers of markers
Posted: 19-Sep-2012 17:07   in response to: sacapuce
  Click to reply to this thread Reply

> <svg id="OpenLayers.Layer.Vector_58_svgRoot"
> viewBox="0 0 900 700" width="900" height="700">
> <g id="OpenLayers.Layer.Vector_58_root"
> style="visibility: visible;" transform="">
> <g id="OpenLayers.Layer.Vector_58_vroot"/>
> <g id="OpenLayers.Layer.Vector_58_troot"/>

ok, so it's using svg. Then the question is whether it's creating the features (in layer.features). Try with just a few features to start with, and if that works increase the number, and see if there's some point where it fails.

> I checked the cached area on IE9 and found the JSON
> file (as a http file) containing the 24000 points
> along with a reference to the bbox boundaries so I am

I think you're misunderstanding the BBOX strategy, which depends on the server being set up to handle a geographical bounds. So if the user is displaying say Canterbury it will ask the server for the features within that area/bounds; if the user then pans to Dover, then it sends a new request for those in the changed viewport. If you try using it with a simple http file request, it will download the whole file and the bounding box param will have no effect.

> As for the compatibility mode, it is set to html5 so
> it should be ok (see below)
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
> Strict//EN"
> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml">

er, that is xhtml not html5 :-)

<!DOCTYPE html>
<html>

sacapuce

Posts: 15
Registered: 19/02/10
Re: Displaying large numbers of markers
Posted: 20-Sep-2012 11:28   in response to: prmaps
  Click to reply to this thread Reply

Hi,
I think you are right and I am not using the BBOX strategy in the right way. I tried with a smaller file and it displays the points in IE9 which is good.

However, in IE9 or Chrome, whenever I pan, it seems to send a request to the server but seem to return the whole file rather than a selection based on the BBOX boundaries. Looking at the temp internet file folder in IE9, it seems that the http call is made with the BBOX boundary added at the end of the url so I am not sure what I am doing wrong.

I checked on the openlayer and openspace websites, forums, and other resources and found very few examples of BBOX strategy with numerous point data in place so I could not figure it out using examples. I have tried all sorts of alternatives by looking at the openlayer index and functions but with no luck.

I have copied the code I am using. If you have any time to have a look at it, let me know what I am doing wrong. I am trying to enquire with the web provider to see whether their server can handle geographical bounds. Again, I am not sure they will understand what I mean. It is all about using the right terminology I suppose.

Thanks
Marc

//create new map
osMap = new OpenSpace.Map('map');

//add vector file
vectorL = new OpenLayers.Layer.Vector( "Vector Layer");
osMap.addLayer( vectorL );

// Modify the OS-given vector layer to add markers from file with BBox strategy

// 1) Define the strategy
var BBox = new OpenLayers.Strategy.BBOX( {layer:vectorL} );
vectorL.strategies = [ BBox ];

// 2) Define the protocol
vectorL.protocol = new OpenLayers.Protocol.HTTP
(
{
url: "RHS4OSO.json",
format: new OpenLayers.Format.GeoJSON()
}
);

// 3) Activate the strategies
BBox.activate();
vectorL.styleMap = new OpenLayers.StyleMap( featureStyle );
vectorL.visibility = true

//set map centre in National Grid Eastings and Northings and select zoom level 5
osMap.setCenter(new OpenSpace.MapPoint(377380, 421897), 5);

prmaps

Posts: 170
Registered: 28/10/08
Re: Displaying large numbers of markers
Posted: 20-Sep-2012 15:09   in response to: sacapuce
  Click to reply to this thread Reply

http://openlayers.org/dev/examples/strategy-bbox.html uses Flickr's api - though doing bbox selection at a country level is not a very sensible example, better to zoom in to a smaller area.

The bbox strategy simply converts the current viewport coords into a bbox param which is sent to the server. Your server program must know what to do with it, and will normally be some sort of front-end to a spatially-enabled dbms. WFS servers will do this, and most of the main dbms these days have spatial features, either specifically like PostGIS or Spatialite, or as an addon, for example http://dev.mysql.com/doc/refman/5.0/en/functions-for-testing-spatial-relations-between-geometric-objects.html

I use the python program featureserver (sadly not really supported any more) as a front-end to a standard SQLite file, which can be used on any web host which has python/sqlite support.

If you don't have any capability like this on your server, there's no point in using the BBOX strategy. You will have to find some other criteria for splitting your data into smaller files, and then requesting the one(s) you want.

sacapuce

Posts: 15
Registered: 19/02/10
Re: Displaying large numbers of markers
Posted: 20-Sep-2012 18:39   in response to: prmaps
  Click to reply to this thread Reply

Ok, thanks for the info. I did not realise the BBOX strategy required specific server features which would have saved me a bit of time.

I managed to develop some code to dynamically load markers based on zoom level and the visible map boundaries.

I basically loaded all markers in a coma separated text file with for each marker a line with eastings, northings and marker info (e.g. 432757,488152,Site number: 14348) which I located on the root directory.

Then I identify the map boundary and parse the file and only select the markers falling within the viewable map. Whenever the user zooms in or out or pans the map, I clean the markers layer and repeat the operation. I set a zoom limit so that at large scale, the markers are not loaded.

I also added a little functionality so that the markers icon size changes according to zoom level.

The code was tested on IE9 and Chrome and it works quite well with a large number of markers.

The full code can be found at the following address: http://www.riverhabitatsurvey.org/DynamicMarkerLoading.js

To see what the map looks like, go to http://www.riverhabitatsurvey.org/?page_id=502

Message was edited by: sacapuce January 2013

Message was edited by: sacapuce

Legend
Expert: 51 - 1000 pts
Advanced: 31 - 50 pts
Intermediate: 16 - 30 pts
Novice: 5 - 15 pts
Newbie: 0 - 4 pts
Helpful Answer
Correct Answer

Point your RSS reader here for a feed of the latest messages in all forums