Gecko dotNET release thread (version 0.65 now!)

[dcx2]

test build r84

- Conditional blr's (beqlr/bnelr/bgtlr/etc) now un-indent the log
- bctrl now indents the log
- b branches now also insert a row of ... into the log to highlight the changing address
- When indenting in the log, there is now one column of |'s per indent, to help keep track of depth
- Step Over and Step Out now preserve the log's indent
- Step Until will clear the Active Condition Checkbox after it is done stepping so you don't accidentally leave the checkbox checked and miss a breakpoint

- Conditional blr's are now caught by Step Out
- Gecko.NET will try to quietly reconnect once before prompting the user to reconnect.  this probably doesn't work on searches...yet...but it will work for Step Until
- Added MemView modes: Single (shows everything as a float), and Auto (tries to guess what it should show the value as).  Auto Zero will show 0 as 00000000, while Auto Dot will show 0 as . . . .

Seriously, the Auto MemView Mode is pretty cool.  Hex, Singles, and ASCII all in the same Memory Viewer screen.  I personally prefer Auto dot mode.

[James0x57]

Well that's a cool idea! That'll be fun to see when all of them are in the window, and especially helpful when hacking objects.

[Panda On Smack]

yeah, nice feature, will test it out.

[IRS]

betta-y goodness (yum)

already having to figure out which i like more.. GUI looks a little more streamlined and graphically "pleasing" but just with a preliminary overview.. i can easily see i will be using both extensively!! favorite thing ive noticed so far.. the ability to sort (by value) the various address results ie. the values each address holds! very helpful for getting fast general ideas of what to look for and not to look for if you know a range ;)

[dcx2]

Okay, so the next feature to develop is improving the search support.  My test case is to pause the game so no memory can change, and then do an Unknown Search, and then an Equals search, first on MEM1 and then on MEM2.  The MEM2 search will create the largest possible files, and Gecko.NET must be able to handle files this large.

If you've ever tried to work with very large search results, you might have noticed that it's a little slow, especially if you tried to save a search to a file.  As it turns out, Gecko.NET reads the dump from Gecko OS into a MemoryStream.  It then does the compare against the MemoryStream (or MemoryStreams, is you're comparing against last value), and creates a List<> of type SearchResult, which has fields for address, oldValue, and newValue.

Serializing a List<SearchResult> to a file of a complete MEM1 unknown-equals search results in a file that is 159 megabytes.   :eek:  That is huge...no wonder saving a large search is slow.

So I tried looking into compressing the stream before writing it to disk.  The results were good and bad...my dual-core Windows XP machine with 2 GB of RAM took 20-30 seconds or more to compress the results and write them to disk, with CompressionLevel set to BestSpeed...but the file size was 48 MB, which is an incredible compression ratio of more than 3:1!

This research leads me to believe that the problem is the List<SearchResult>.  The data structure is just too large.  It's also redundant; most of the time, the MemoryStream that has the search dump is still lying around while there are duplicate values in the List!  So I think I'm going to partially gut the search function and re-write it.

The plan is to replace the List<> with a standard UInt32[] that just holds the addresses of the successful results.  And instead of copying search results into the List<> I'm going to read the values directly from the original dump, so they're only in one place in memory.  Then, I'm going to populate the search results' ListBox with values derived from the UInt32[] and the dumps.

After this, there are a lot of neat new features that will be easy to implement.  For instance, I will see if I can populate the search results' ListBox while the search is transferring.  I can also make it so that dumps can be paused and restarted.  I can compress the search dump behind the scenes while it's being transferred.  I can then store pre-compressed dumps to disk to provide an Undo mechanism.  I can also store many pre-compressed dumps, to provide a multi-level undo.  The multiple dumps can also act as a multi-level history, so you can not only see NewValue and OldValue, but SecondOldValue, ThirdOldValue, etc.  This also provides opportunities to extend the search comparisons from "specific value" and "last value" to include "first value" or "nth value".

So...this might take a while...and the searching could get a little buggy for a few releases while I work the kinks out of the new system.

[James0x57]

That sounds awesome man! Are you using any specific compression algorithm?

[dcx2]

Whatever DotNetZip uses.  It probably depends on what you set the CompressionLevel to, but I think it uses GZip.  It's free and open source, and that's what I care about the most.   ;D

I would use the built-in GZipStream in .NET but I hear it's really crappy, and it's possible to embed a dll into a .NET assembly using ILMerge anyway so I don't even have to worry about distributing another file.

And DotNetZip is also Mono compatible, for whenever I decide to go for round 2 on the Mono thing...

[hetoan2]

Auto MemView is amazing! Very helpful when the game uses ascii in the code. In the Call of Duty games you can actually get C++ code out of the memory :P

It would be helpful if I could upload stuff using geckoDotNET though... WiiRD can do it (not GUI). So why cant you make on option to open the console? It'd be very helpful if I could dump and upload the modified C++ files from the games :) *hinthint*

[dcx2]

Eventually I'd like to color-code cells according to their suggested data type.  However, I think Auto Mem View will probably need some tuning.  Here's how it works.

Anything that's a valid pointer will be shown as a hex value.

Anything that's not NaN, and whose absolute value is greater than 1e-7 and less than 1e10, will be shown as a float.

Anything that has four valid ASCII characters (0x00, 0x20 -> 0x7F) will be shown as ASCII.

Everything else is shown as hex.

The tuning would involve modifying the upper and lower bounds for what is considered a float.  I chose those numbers because they're smaller/larger than the practical values that you normally find floats used for; some games probably use three or four digits past the decimal, but not seven.  Also, some games use millions or billions, so I set the float cap at a trillion.

I think it also needs some ASCII tuning, too.  For instance, four valid ASCII characters, but 0's cannot be a leading character, only trailing.  So 0x39310000 would be treated like ASCII, but 0x00393100 would not.

EDIT: oh yeah, hetoan, look at Gecko.NET's memory viewer's context menu.  There's an Upload menu item that I never looked at.  It might do what you're asking.

[Mokuro]

Any chance getting pointer search for Gecko.Net ? A separate program will do if pointer search is not possible to be added to our lovely Gecko.Net. I just can't stand Wiird anymore. Crashes like crazy.  8)

[hetoan2]

1e-7 is a little too small.

I have never seen a floating point used that was below 1e-4

The hexadecimal cod 35320000 gives a floating point of 6.6e-7 even though it is for the ascii "52"

Also in the very first area of mem1 there are values stored to certain addresses that always hold a certain value. Like at $8000000 the title ID is stored and at other values stuff like FST and whatnot is stored. These addresses always have the same type of value, so i'm suggesting that for those areas that they always be shown how they were meant to (ascii for ID and plain hex for FST, etc.)

Here's a link to the memory map in case you want to implement this (or fix the FST :\)
http://www.wiibrew.org/wiki/Memory_Map

Also thanks for telling me about the upload function, it should be helpful. Although command lines are a lot easier to copypasta from a txt document :(

[dcx2]

Ideally, there would be a separate file which stores the data type of each memory viewer cell.  So that way, if auto guessed wrong, you could tell it that it's wrong.  But that might be a bit complicated.

[wiiztec]

Will the color coding be optional? there's plenty of room on the tools & about tabs to put some options, and it might be distracting to some people

[dcx2]

wiiztec - Color coding, if I ever did it, would be optional.  I might even make it programmable, if you don't mind trolling through a text config file and changing values there.

Mokuro - I don't think I will ever personally work on a pointer search mechanism.  I don't see the point, really, when you can just use a read or write breakpoint and find the offset without wandering through various levels.  Or when you can use a C2 hook and get around the whole pointer problem.

After re-working the search code, my next goal will probably be integrating PyiiASMH-like support into the GCT Wizard.

hetoan - I will consider making the thresholds for float programmable (probably via text config file).  I wasn't sure what the smallest/largest practical float is, so I decided to go with things that were a few orders of magnitude smaller/larger than the smallest/largest things I've ever personally seen.

I will also consider more ways to appropriately recognize ASCII (for instance, "is the previous memory word ASCII?  If so, this memory word is more likely to be ASCII")

[hetoan2]

sounds like it'll be good. txt config files for colors and floats is good with me :D