A better way to handle texture scale/cache

[RadarNyan]

I don't actually understand how PSP works, so the following are most my own opinion - basically guessing. so correct me if I'm wrong.

(And... excuse me for making this post so long... English is not my first language I've no idea how I can make it shorter.)

---

I've been messing around with PPSSPP's texture cache/scaler for quiet a few days, trying to get more textures scaled and reduce the rescale on the same texture (it happens a lot, I'll explain later)

The problem with PPSSPP's current methods is, it doesn't scale the "real" texture, instead, it gets which address the texture is loaded in ram, and grab the texture size of ram after that address (let's call this "texture-ram" from now for convenience)

This works fine for most circumstances, but it leads to a problem that some times the unused part of texture-ram is reused for other usage, mostly when displaying a 480x272 picture, it takes a 512x512 texture and the rest part of texture-ram is not relevant to texture, could be used for anything else as available memory - when it's being used, it may very well be different any moment (even within a single frame)

So here comes the problem: currently, we're hashing from texture-ram, the changing of the unused part in texture-ram leads to the changing of hash, so we got the different hash of the exactly same texture and consider it has been changed - then the cache of this texture is removed, and generated again.

This is a total waste of resource, it may not be so noticeable if you don't enable texture scale feature since we often run PPSSPP on platforms far more powerful than a real PSP, we can handle it anyway. But if you enable texture scale, you may very well notice the performance drop caused by this design flaw in a lot scenes in a lot games (but you didn't, I'll explain why right away)

So PPSSPP comes up with a solution of not scaling textures updating too frequently and limit the amount of textures being scaled per frame. That's why you didn't notice the performance drop I mentioned in the past paragraph and you may very well notice that some textures are not scaled as you intended to.
(current limit from the latest git build is if a texture updates more frequently than once per 6 frames, don't scale it. If more than 256x256 of textures is scaled this frame, no more scaling this frame)

---

(Again, I apologize for being wordy x_x I hate myself about this as well as you might right now.)

What I'm thinking is, when a texture is being loaded:

(Pseudo-code, @variable, $scaleFactor is a variable contain info of scale level, scale method... used for the "key" of cache)

Code:

@hash = hash(texture1)
if ( TextureCache[@hash].key == $scaleFactor ) {
    // cache hit, return from cache
    return TextureCache[@hash].data
} else {
    // no hit, store it into cache
    @textureData = ScaleTexture(texture1)
    TextureCache[@hash].key = $scaleFactor
    TextureCache[@hash].data = @textureData
    return @textureData
}

So basically, I want to move the hash part from memory to when the texture is actually being loaded, but so far I have no idea where the loading part actually is and I don't really have enough programming skill to achieve what I'm thinking...

unknownbrackets and hrydgard has explained a lot to me in this issue #6256 on github but I don't really get everything they said (stupid me x_x)
You may want to check that issue (sorry I don't have the permission to post links) if you don't understand what I said above, there're some screenshots in it.

[[Unknown]]

The basic problem here is that textures are loaded from memory. What else are you going to hash?

If you want to try to figure out when a game is loading textures based on file access or something else, good luck. At best you will end up with an emulator which only supports one or a few games from the same developer, and has lots of problems with any other games.

The PSP itself has no special mechanism for loading textures like OpenGL does. From its perspective, it just needs to know:

Where does the texture start in RAM? __________
What is the size of the texture in texels? ___ x ___ (must be powers of two)
How many texels between each line? ____ (need not be a power of two)
What format is the texture in? _________
Is the texture using a CLUT? [ ] Yes, its format is ___________
Do I need to load the CULT? [ ] Yes, its address is __________
<many other settings dealing with clamp/wrap, UV handling, etc.>

The PSP textures directly from memory. This is great, and why it doesn't need to deal with the problems we have. Unfortunately, there's no way to tell OpenGL to texture directly from memory - we have to provide it with the full texture and update the texture when the memory changes.

Additionally, the PSP supports CLUTs and compressed formats not supported in GLES 2. So we have to also decode these and send them to OpenGL. This means even if it could texture directly from RAM, it couldn't work for these (very common) textures anyway.

It's true that games probably "load" this memory from other places: sometimes other RAM, sometimes files, sometimes functions that generate the texture. All of this is highly game specific (or at least game engine specific.) And, you'll never catch every case.

-[Unknown]

[Henrik]

We could keep track of the max and min UV coordinates that have been used so far when texturing from each specific texture, and only hash that subrectangle. That would add quite a bit of complexity and overhead though and will not work for env mapping (texture projection). So I'm really not sure it would be worth it.

[[Unknown]]

Right, at least not for most textures. For simple full screen ones it might work out, but it may cost and slow down games like Tales of Phantasia X which hit the software transform path a lot and generally don't have this problem.

If the goal is just higher definition textures, maybe a better option is texture replacement. We could have definitions that allow e.g. a texture at a specific address to be replaced, or even to specify the size of bytes at that address to hash. That is:

Code:

auto replacements = replaceMap.find(cacheKey);
if (replacements != replaceMap.end()) {
    std::map<u32, u32> hashes;
    for (auto replacement : replacements->second) {
        if (hashes.find(replacement.size) == hashes.end()) {
            hashes[replacement.size] = ReplaceTexHash(texaddr, replacement.size);
        }
        if (hashes[replacement.size] == replacement.hash) {
            SetReplacedTexture(replacement);
            return;
        }
    }
}

But that would naturally be a very high-maintenance (for humans, not meaning code) way of maintaining replaced texture sets.

-[Unknown]

[RadarNyan]

IMO, getting every subrectangle and hash them separately is totally overkill.

Since the "random" data could only shows up at the end of texture (in ram), if it's possible to get the last line being used before, just hash from the beginning to this line will be totally solve the problem isn't it?

   
Is it possible to get the last pixel of the drawing area?
Actually it's not necessary to get last pixel, just the last line will be enough. Then we just keep update the last line (of the same texture, of course) if a new drawing area's last line is bigger than current value, at last we will end up with a line that actually been used from this texture.
(well, this texture doesn't need this method at all... Just grabbed a random one)

And if we can manage to get this "correct range", is it possible to only scale this range of texture? Because those "random" data is really really slowing down the texture scale process.

But the problem is, how can we get this unique id of a texture? A same address could be totally used by different textures right?

[[Unknown]]

The problem is, consider a game using the texture you screenshoted above. I have seen tons of games use textures like that for 3d models.

In that case, it's obvious (to a human) that the entire texture is actually a texture. Hashing the whole thing is right. Here's what happens if you don't hash the whole thing:

1. Game uses top half of texture, ending 50% into texture Y.
2. We calculate the hash as 5123456.
3. Game later (after using another texture, maybe) uses the other half of the texture (going as low as you showed.)
4. Now we see its sizeInRAM as different, so even though the data is identical, we calculate the hash as 2344576.
5. We reload the texture into OpenGL unnecessarily.
6. Go back to 1, where we will again reupload.

This would kill framerates. At best, it would need to track the highest value used in the texture recently. Even then, it will slow down the initial draw of screens (like we saw with Type-0 recently) since it would re-upload textures like I described above for the first frame.

In my example, I meant that texture replacements would say how many bytes of RAM they thought the texture is, therefore allowing the issue to be worked around manually.

The same address can and definitely is used for completely different textures. The only way we get any sort of unique id right now is by hashing it.

-[Unknown]

[RadarNyan]

What if we keep a copy of the original (not processed) texture data (in the full power-of-two size) when we do the process (scale.. etc)

Then when the sizeInRAM changed, we compare the hash of current texture data (using sizeInRAM) and the old copy data (also use sizeInRAM) if they are identical, just use the cached texture instead of reloading it.

This way we could avoid uploading the same texture too frequently so we avoid a terrible performance drop caused by reloading textures.

But... since hashing will be become way more frequent than the current method, will it cause a performance problem? (I guess not?)

It wouldn't take too much extra memory if we keep a copy (of original data) right? I mean... If we can keep 4xscaled textures (which I assume at least three times bigger than original texture) in memory, sure wouldn't be a problem just for double original size, right?

[YaPeL]

(06-18-2014 11:04 PM)[Unknown] Wrote:  Right, at least not for most textures. For simple full screen ones it might work out, but it may cost and slow down games like Tales of Phantasia X which hit the software transform path a lot and generally don't have this problem.

If the goal is just higher definition textures, maybe a better option is texture replacement. We could have definitions that allow e.g. a texture at a specific address to be replaced, or even to specify the size of bytes at that address to hash. That is:

Code:

auto replacements = replaceMap.find(cacheKey);
if (replacements != replaceMap.end()) {
    std::map<u32, u32> hashes;
    for (auto replacement : replacements->second) {
        if (hashes.find(replacement.size) == hashes.end()) {
            hashes[replacement.size] = ReplaceTexHash(texaddr, replacement.size);
        }
        if (hashes[replacement.size] == replacement.hash) {
            SetReplacedTexture(replacement);
            return;
        }
    }
}

But that would naturally be a very high-maintenance (for humans, not meaning code) way of maintaining replaced texture sets.

-[Unknown]

Looks like a nice functionality to add to ppsspp, does dolphin do it that way too? is someone working on it?

[Arborea]

Would be possible to inject for example depth maps or displacement maps for Parallax Occlusion Mapping/Displacement Mapping/Tessellation or it requires full texture replacement ? Or even lightmaps , environment maps , shadow maps for more advanced lighting/shadows ?

[RadarNyan]

Let me describe what I have in mind. All codes are pseudo-code.

1) A texture (let's say 0x09658640, 128*128, format 3) is being used to draw something on screen

We use the information of this texture that is "reliable" (well, more reliable than RAM hash) in this case, let's say "0x09658640_128_128_3". We can make it shorter by hash it if we want, or just use it anyway. I'll use "KEY" to represent "0x09658640_128_128_3" in the following description.

Quote:Let me descript the TextureCache design of mine first for the following description:

TextureCache[KEY][0]->TEX : Processed texture (color convert, scale, etc) like we currently have in cache method.
TextureCache[KEY][0]->RAW : RAW memory data from the entire 128x128 range, don't care if it's all texture or not.
TextureCache[KEY][0]->HASH(array) : An array contains several hash that matches this cache entry.

Since a same address could be totally used by different texture, a INDEX ("[0]") is needed.

2) We create a hash of this texture

But we don't hash the entire texture (128x128) like we're currently doing.
Instead of hashing to the end (line 128), we only hash to the last used line (let's say 100).

Code:

HASH1 = DoQuickHash(DATA_MEMORY, 128, 100, format 3)

(DATA_MEMORY is what we think the texture is in memory, which is the thing we're hashing in current method)

3) We check if we have a cache hit

Code:

Loop {
    _INDEX_ = 0

    for _HASH_ in TextureCache[KEY][_INDEX_]->HASH(array){
        if _HASH_ == null { // We reached a "empty" spot, which means we don't get a cache hit
            break the Loop
        } else if _HASH_ == HASH1 { // We got a cache hit
            return TextureCache[KEY][_INDEX_]->TEX
        }
    }

    // Don't give up yet! (this is why we keep copies of RAW data in TextureCache)
    HASH2 = DoQuickHash(TextureCache[KEY][0]->RAW, 128, 100, format 3)
    If HASH2 == HASH1 { // We got a cache hit
        // Add this HASH to the array
        TextureCache[KEY][_INDEX_]->HASH(array).add(HASH1)
        // Return the texture
        return TextureCache[KEY][_INDEX_]->TEX
    }

    _INTEX_++
}

4) Process the texture, make a new texture entry

We can only reach this step if we do not have a cache hit, and by the time it breaks the loop, _INTEX_ will be the INDEX we should use for our cache entry. let's just use 1 in the following description, it will be 0 if it's the first time the texture is loaded of course.

Code:

TextureCache[KEY][1]->RAW = DATA_MEMORY
DoProcessTexture(DATA_MEMORY) // I'm assuming DoProcessTexture function modified original data
TextureCache[KEY][1]->TEX = DATA_MEMORY
TextureCache[KEY][0]->HASH(array).add(HASH1)

5) Return the processed texture

Code:

return DATA_MEMORY

Pros

1. Getting much more cache hit than current method

2. Kind of having a unique ID of a texture, we may use it for texture replacing, or save texture cache to disk for speed up.

Cons

1. DoQuickHash method is called when every single draw, once or twice

Which is way more frequently than what we're currently doing - only hash once while binding a texture.

I'm not sure how fast this DoQuickHash method is, maybe we need something faster, or this very frequent hashing might kill us.

2. Some memory can not be purged if we want to get the unique ID correctly

We can clear TextureCache[KEY][INDEX]->TEX since we can always regenerate it from TextureCache[KEY][INDEX]->RAW, but TextureCache[KEY][INDEX]->RAW and TextureCache[KEY][INDEX]->HASH(array) must not be purged or we will get wrong ID (wouldn't be a problem if we don't want this ID though)

This can be solved if we write the TextureCache to disk, Something like:

Quote:0x09658640_128_128_3_0.png (processed texture)
0x09658640_128_128_3_0.raw (RAW memory dump)
0x09658640_128_128_3_0.txt (contains all possible hash to this texture)

so we can load it back as needed, and if we want to do Texture Replace (otherwise why would we need a unique ID?) we need a dump function anyway. We do not have to manually make texture packs, just dump the textures been processed by xBRZ will be a huge success. Well, it's more like a on-disk texture cache instead of a texture pack, but texture pack is also possible since we have the unique ID.

3. We sure will need more memory

Let's say if a game have 1GB of textures, we will need at least 2GB (1GB RAW + 1GB cache, and we know the cache can't be just 1GB, if we use texture scale it will be much bigger) memory for this unique ID to work (well, it won't be needed right at the beginning, but sure will reach at least 2GB eventually) Most desktop PCs these days doesn't hurt from this at all but all mobile device will suffer from this huge memory hog.

This can be solved if we don't want the unique ID anyway - just purge as will. This method will be working just fine for real-time (as long as we don't suffer from the way more frequent hash calculations) truth if we purge the TextureCache, we sure will loose some possible cache hit, but I believe the hit rate is still better than current method.

This can also be solved with the solution of "purge"-to-disk or cache-on-disk, once we need it, we can purge both TextureCache[KEY][INDEX]->TEX and TextureCache[KEY][INDEX]->RAW. TextureCache[KEY][INDEX]->HASH(array) wouldn't take too much memory anyway so we don't even need to think about it.