Gigantic map data structure

[ooflorent]

I'm working on a game which has very very large worlds (billions of tiles).

Storing such data into an Array isn't possible and I'm trying find the best data structure to fit with my needs.

 

How should I store and read the world map?

I took a look to the File API but the solution didn't appeared to me...

Maybe something with a FileReader. Any clues?

[Quetzacotl]

I would download map data from cloud server when it's needed. Downloading it whole at once on game load is too much.

 

You can make your own server api and download data using ajax or you can just download static json files from server when you need them.

In example split map data into bigger tiles, like 10 000 x 10 000 and save it into json files named 0x0.map, 0x1.map, then get it as json using ajax when player needs it.

[Antylum]

    What approximately size of data in mega/gigabytes do you want to store? 

If it about 5 mb, than you can use local storage, if more, here is the solution which uses appCache(see links), it's supported by more browsers than WebSQL or IndexedDB. Also it depends on what type of data you need to store(graphics/sounds or only javascript objects). Here is more detailed information about storing data on client-side: 

http://blog.sklambert.com/a-comprehensive-guide-to-taking-your-html5-game-offline/

http://diveintohtml5.info/storage.html#limitations

 

I think the best way(if you have tens, hundeds megabytes of data) is to use server, and separate map into regions. Client's game logic must control player's position, and download new resources or hole region, if player steps into new world, or opens new chapter. Write detailed information about what you need.

[xerver]

I agree with Quetzacotl, I would stream "chunks" of the map to the client and store them in IndexedDB as a cache. That way you could load from IndexedDB instead of the server next time, but you also don't have to keep each chunk in memeory.

[ooflorent]

Thanks everyone. I'll try these approaches.