My map load parser is taking ages on android

[tommohawkaction]

I am having some trouble with loading on android as it is taking for ever to load a 500x500 map

Okay first:
How do i make it load from another thread? I am using libgdx but I would like to know how to do it in pure java and libgdx Gdx.app.postRunnable() I want another thread so i can show the user that the game is loading with a little animation screen

second for a map size of 500 x 500 should i consider using the chunking system… if so how?

Here is my parser code

public static TileMap loadMap(FileHandle file) {
		if (!file.extension().contains("hawk")) {
			System.err.println("Wrong file");
			return new TileMap(100, 100, TileType.GRASS);
		}
		TileMap map = null;
		BufferedReader reader = new BufferedReader(new StringReader(file.readString()));

		try {
			String[] configString = reader.readLine().split(",");
			int mapWidth = Integer.parseInt(configString[0]);
			int mapHeight = Integer.parseInt(configString[1]);

			String line = null;
			int counterX = 0;
			int counterY = 0;
			
			MasterTile[][] tiles = new MasterTile[mapWidth][mapHeight];
			
			while ((line = reader.readLine()) != null) {
				String[] split = line.split(",");
				int botTile = Integer.parseInt(split[0]);
				int topTile = Integer.parseInt(split[1]);
				float rotationBot = 90;
				float rotationTop = 90;
				boolean fXBot = false;
				boolean fYBot = false;
				boolean fXTop = false;
				boolean fYTop = false;
				if(split.length > 2){
					rotationBot = Float.parseFloat(split[2]);
					rotationTop = Float.parseFloat(split[3]);
					if(split.length > 4){
					fXBot = Boolean.parseBoolean(split[4]);
					fYBot = Boolean.parseBoolean(split[5]);
					fXTop = Boolean.parseBoolean(split[6]);
					fYTop = Boolean.parseBoolean(split[7]);
					}
				}
				Tile bot = TileUtils.createTile(botTile);
				bot.setRotation(rotationBot);
				bot.setFlipX(fXBot);
				bot.setFlipY(fYBot);
				Tile top = TileUtils.createTile(topTile);
				top.setRotation(rotationTop);
				top.setFlipX(fXTop);
				top.setFlipY(fYTop);
				
				tiles[counterX][counterY] = new MasterTile(counterX*MasterTile.TILE_WIDTH, counterY*MasterTile.TILE_HEIGHT, top, bot);
				counterY++;
				if(counterY > mapHeight-1){
					counterY = 0;
					counterX++;
				}
				if(counterX > mapWidth-1){
					counterX = 0;
				}
				
			}
			
			map = new TileMap(mapWidth, mapHeight, tiles);

			reader.close();
		} catch (IOException e) {
			e.printStackTrace();
		}
		

		return map;
	}

[Riven]

@tommohawkaction

 while ((line = reader.readLine()) != null) {
            String[] split = line.split(",");

This is a regex split, which is a very heavy operation. You’ll have to check whether that is the culprit…

What is your loading time currently? ‘forever’ is not really helping.

[tommohawkaction]

@Riven

@tommohawkaction

 while ((line = reader.readLine()) != null) {
            String[] split = line.split(",");

This is a regex split, which is a very heavy operation. You’ll have to check whether that is the culprit…

What is your loading time currently? ‘forever’ is not really helping.

On computer pretty much instant
On android your talking about min 5 seconds to 40 seconds

[Cero]

Hm wow.
How big are your map files?

What exactly is involved in doing new MasterFile() ?
Also TileUtils.createTile

[trollwarrior1]

If you are using Eclipse, there is a Trace View in DDMS Layout. Look at answer by Seppl

how to use traceview in eclipse for android development?

It will show you how much time was spent executing parts of your code.

[tommohawkaction]

@Cero

Hm wow.
How big are your map files?

What exactly is involved in doing new MasterFile() ?
Also TileUtils.createTile

9.29 MB

	public MasterTile(int x, int y, int topTileID, int bottomTileID) {
		topTile = TileUtils.createTile(topTileID);
		bottomTile = TileUtils.createTile(bottomTileID);
		this.x = x;
		this.y = y;
	}

public static Tile createTile(int ID){
		// Some are static because they do not have individual movement (Animations etc...)
		// This is where I create all of the tiles using IDs or providing it with a static tile which is located in the BlockType.class
		switch(ID){
		
		// Tiles
		case AIR: return AIR_TILE;
		case GRASS: return new GrassTile();
		case DIRT: return new DirtTile();
		case GRAVEL: return new GravelTile();
		case SAND: return new SandTile();
		case WATER: return new WaterTile();
		case PLANK: return new PlankTile();
		case TILEDWOOD: return new TiledWoodTile();
		case STONE: return new StoneTile();
		case BRICK: return new BrickTile();
		case REDBRICK: return new RedBrickTile();
		case CHECKERED: return new CheckeredTile();
		case CHECKEREDBLUE: return new CheckeredBlueTile();
		
		// Objects
		case TREE: return new TreeTile();
		case TORCH: return new TorchTile();
		case FIRMTREE: return new FirmTree();
		case RFLOWER: return new RedBrickTile();
		case PIFLOWER: return new PinkFlower();
		case PUFLOWER: return new PurpleFlower();
		case BFLOWER: return new BlueFlower();
		case BUSH: return new BushTile();
		case DEADTREE: return new DeadTreeTile();
		case WOODENDOOR_CLOSED: return new WoodDoorTile(false);
		case WOODENDOOR_OPENED: return new WoodDoorTile(true);
		default: 
			System.err.println("(TileUtils) Tile doesn't exist!");
			return AIR_TILE;
		}

[StrideColossus]

I assume the FileHandle parameter is a LibGDX data-source type thing? Presumably the readString() method loads the entire file into a string before the parser code even starts, which may be an issue on a memory-starved device but not on a PC.

i.e. Try changing:

BufferedReader reader = new BufferedReader(new StringReader(file.readString()));

to:

BufferedReader reader = new BufferedReader(file.reader());

But as previous posters have said a bit of tracing should identify the culprit(s) and/or step through the code using the debugger to see what’s taking the time.

[tommohawkaction]

@Brynn

I also think some optimizations could be made to how your game is. I believe you have an instance of a tile wherever there is a certain type of tile (if that made sense), Perhaps you should have one instance of every tile and store an identification number (per tile) in order to reference it? Sort of like how minecraft and many other similar voxel box games go about it. 9 MB seems like a lot for such a simple save file. Could we see the level saving too?

Ah okay I see, it would also work with each block having it own rotation… (Never thought of it like that) thanks

[Abuse]

Is there a reason you’re using human-readable map files?

Loading 4 bytes into an int is a lot more efficient* than loading half a dozen chars into an array, then parsing them into an int.

*smaller data files, lower peak memory consumption & faster execution.

[tommohawkaction]

I have no idea how to

[Opiop]

You have no idea how to what?

[tommohawkaction]

@Opiop

You have no idea how to what?

I have no idea how to [quote]Loading 4 bytes into an int is a lot more efficient* than loading half a dozen chars into an array, then parsing them into an int.
[/quote]

[chrislo27]

This code I have from my current project can read/write an array of bytes to a file and GZIP (compress) it.

public static void saveBytes(byte[] world, File location) throws IOException {
		FileOutputStream fos = new FileOutputStream(location);
		GZIPOutputStream gzipstream = new GZIPOutputStream(fos);

		gzipstream.write(world);

		gzipstream.close();
		fos.close();
	}

	public static byte[] loadBytes(File location) throws IOException {
		FileInputStream fis = new FileInputStream(location);
		GZIPInputStream gzipstream = new GZIPInputStream(fis);

		ByteArrayOutputStream byteStream = new ByteArrayOutputStream(2048);
		byte[] buffer = new byte[2048];
		int bytesRead;
		while ((bytesRead = gzipstream.read(buffer)) > 0) {
			byteStream.write(buffer, 0, bytesRead);
		}

		gzipstream.close();
		fis.close();

		return byteStream.toByteArray();
	}

If you’re not in need of compression you can replace the GZIP streams with a FileInput/OutputStream instanceDataInput/OutputStream and it’ll read/write uncompressed files.

EDIT: Riven said using Data(Input/Output)Stream is for binary data. I just assumed FileInput/OutputStream would work.

[tommohawkaction]

How would you do this in libgdx using a string?

[Cero]

well byte to string and vice versa is not a problem

but 9mb map files geez, unless you have assets IN your map file, which I will assume here…

and calling createTile 3 times per tile… which calls one of these methods, which we dont know how intensive they are

[tommohawkaction]

@Cero

well byte to string and vice versa is not a problem

but 9mb map files geez, unless you have assets IN your map file, which I will assume here…

and calling createTile 3 times per tile… which calls one of these methods, which we dont know how intensive they are

You didn’t answer my question?
How do i change string to a byte

[BurntPizza]

http://docs.oracle.com/javase/7/docs/api/java/lang/String.html

string.getBytes();
new String(byte[] bytes);

[Opiop]

@tommohawkaction

@Cero

well byte to string and vice versa is not a problem

but 9mb map files geez, unless you have assets IN your map file, which I will assume here…

and calling createTile 3 times per tile… which calls one of these methods, which we dont know how intensive they are

You didn’t answer my question?
How do i change string to a byte

There’s something called finding it out on your own…

[Riven]

@BurntPizza

http://docs.oracle.com/javase/7/docs/api/java/lang/String.html

string.getBytes();
new String(byte[] bytes);

specify a charset or be ready to deal with unexpected behavior. besides that, the string class should not be used for abitrary binary data, only encoded charater data. even utf-8 cannot be used for storing and retrieving arbitrary binary data. use base64 for such purposes.

having said that, use DataInputStream / DataOutputStream if you’re going to use a binary format.

[Cero]

@tommohawkaction

@Cero

well byte to string and vice versa is not a problem

but 9mb map files geez, unless you have assets IN your map file, which I will assume here…

and calling createTile 3 times per tile… which calls one of these methods, which we dont know how intensive they are

You didn’t answer my question?
How do i change string to a byte

Because its not the issue and as Riven pointed out very suboptimal anyway.
Start by not having 9mb files. Unless your game is just THAT complex