Stephen's Weblog

I continue to fail to post weekly; all the fun stuff I would want to say, I don’t think I should talk about, though.

Anyway, the Opti compo is still going along, but it’s getting close to the end – right now there are 11 days before the compo ends. I’m adding this informational post to remind people and also to alert people to a new revision of the test system, better rules for whether an entry will be accepted or not, and various other things. I’ll reiterate  the important parts on the competition page.

Firstly, the test system update: this update will probably be the final update of the test system code for this competition. Unfortunately I didn’t get all the stuff I wanted in it, I’ve been short on time and have had more than enough stress to go around :) So, this update does not contain the statistical time-sampling profiler that I wanted to add to the test app. It also doesn’t include a fractal explorer section which I had wanted to add. But on to the features that I did get around to adding: There is now a new default test set of 6 tests, they zoom in quickly on a specific point in the fractal, and are pretty complex areas to render (the number for max_iterations on the 6th image is 1400, and the horizontal and vertical steps are in units of 2^-44). I’ve included a modified example that is designed to serve as a lower bound for correctness – passing implementations should have a smaller amount of error than the lower bound. I don’t think this will be too hard, the lower bound currently exists as a modification to the multiply code that loses 3.5 bits of precision with each multiply. This actually caused less difference from the reference implementation than I had thought; currently across the 6 tests, there is a total difference of 7044 iteration counts, notably the 6 tests render a total of 147456 pixels, so this value is really barely even noticeable.

There was also a bug in the test app that caused it to only display the last test’s cycle count rather than the sum of all of them, this has also been fixed, and (in no$) the cycle count for the “lower bound” test is 0x0004CC483E1E cycles – which is 307 seconds of cputime, about 5 minutes. As the default test behavior also runs the example code alongside it, this means a test run of the lower bound takes about 10 minutes; not very good for repeated testing. I’ve added a way to start the tests without checking them, by pressing start instead of A, so the code execution time you have to sit through is purely your own fault ;) It will be pretty obvious if there are big errors, but it will still take a long time to check results against the reference function.

Ok, that’s all I have for now; I’ve been working on my own implementation just lately in assembly, but I really have no idea how well it will perform just yet. I’ll post some numbers in the comments on the Opti site once I get them.

Ok, Finally – after a lot more effort than I thought it would take – the akkit.org optimization contest (for DS) is up and running, with it’s first competition!

Although the competition technically started a day and a half ago, I haven’t posted it yet because I didn’t have the test package ready, but is ready and posted now.

So, it is with great anticipation that I submit it to you, for review: Opti’s First Competition

Anticipation like, waiting for a pie to hit me in the face, or be run over by a car, or something similar ;) I do however want to see what level of interest there is, and try to make this into something that more people will be more seriously interested in next time (I am highly curious about how many people will be interested this first time) – So, to that end I am very interested in any sort of feedback. Anyway – That’s all I want to say for now, I’ll shut up before I ramble on about it for a very long time; Thanks for looking :)

Ok, so the optimization contest thing didn’t make it this last weekend :( sorry about that, I’m overly optimistic and didn’t have time to work on it enough to get it ready.

Now, I’ll take a few moments to relate some of the things that are going on in my life at the moment; as you might have noticed my online presence has been rather very thin, and my projects have been crawling along, if that – and this isn’t just for the projects that are public, even the private ones are not moving much… The culprit I’ve decided is stress, it’s the only logical cause I can come up with.

To that end, I’ve decided to quit my current job – I’ll be working for the next 2 weeks and then I’ve arranged to quit. At that point I’ll probably get some rest for a week or 2, I’m planning to visit some relatives and then I’ll devote some time to getting my currently idling projects done.

I hope I can get the optimisation contest site running in the next week or 2. January though will bring work on the nds_test_apps, probably dstunnel (due to popular demand), and some other things I haven’t figured out the order of yet. So, onwards! We will see what the future brings.

So, things have been getting pretty stagnant lately – I’d really prefer to have more visible progress, I really would – reality is not always kind to me. Either way however, I will try to pick up the pace a bit and get more done on my more public projects… I reiterate that if you would like to help on any of these projects, feel free to contact me.

The project du jour is, however, the NDS test apps project. It is a prerequisite for some other fun things I’d like to do, and it’s also very useful to the community (if I do say so myself) – I’ve set up a new IRC channel for it – on irc.blitzed.org channel #nds_test_apps – feel free to join if you’re interested, have suggestions, want to code for the project, etc. I’m still not entirely sure where these projects are really going to go, but I’d like to get a few of them done if possible, and I really don’t mind the help from interested parties!

The first project I’m going to use this “system” for is a set of testing apps for DS, primarily to verify emulator correctness. The project page is at http://wiki.akkit.org/Nds_test_apps. The page itself describes the project pretty well, but here’s a quick summary:
Two applications for DS will be written, one is a Graphics test, one is a CPU/Memory test. The idea behind both, is to evaluate the correctness of emulation for all graphical and CPU features, and periodically release the applications and information about how well the emulators do in the tests in order to promote improvements in emulation.
This project is nice in that it’s easy to add more users to the development without people stepping on each other’s toes, so I’m happy to add several more people to the project.

Not all of the framework is in place for this project yet. I have a Subversion server and a bugtracker that are both partly ready, I expect to have both of them running correctly soon – they’ll be responsible for making this teamwork possible.
The code for this project will be closed source during development, until a certain level of completeness is reached (this has yet to be decided) – after which point, all the code will be released under the MIT source code license (this, too, is just the current plan, if the team has other ideas, it can change)
We will release binaries periodically, to allow people to test on their own, and to allow emu authors to get a good look at what problems exist.
If you would like to be involved with this project, please email me at sgstair [at] akkit [dot] org.

I’ve put together a really quick and hacky xmassy demo for #dsdev / blitzed and the homebrew community in general :) behold! the NDS file (xmas.nds, 85k). Many thanks to nornagon (in #dsdev/blitzed) for helping me out with the fonts.  Enjoy the computer-generated snowflakes, and have a happy holiday season everyone :)
(also, a screenshot!)

Hi everyone, it’s been a while, but here are my priorities in the homebrew realm for the immediate future. I’m going to disappoint some of you, sorry – but this is how it has to be.

The DSWifi project is going to be discontinued for the time being, until sometime next year. I’m not going to go into more detail on why, but I will not be continuing to work on it for now. I would like to turn it over to an individual or group of people if anyone would like to maintain it – if you’d like to do this, get in contact with me (on IRC, or leave a comment, or send an email…)

I will be starting a new project to reverse engineer the Wii’s wireless hardware, and build a new library based on it. This is expected to start around the start of december, and run a few months (who knows how long it will take)

The DSTunnel project is currently inactive, but I will make an attempt to revive it later this year, I would like to first complete the Wii wifi library, and I may actually build DSTunnel directly for the wii, or at least as one of the supported platforms.

I have 2 undisclosed DS projects that I will be working on… you’ll just have to wait and see what they are :)

Beyond all that, I have a few other general purpose libraries (besides the 802.11 and TCP/IP libs) that I’m going to start on at some arbitrary time (FAT lib, crypto lib, and some others) and a few emulator projects I want to work on.

So, that’s what my schedule looks like for the next several months – I now await your kind comments and harsh criticism ;)

I received an email not long ago indicating that there was a bug in the checksumming code in the lib – and as it turns out there was indeed an instance where the checksum was still being calculated incorrectly – this bug only affects UDP packets, and not very many of them (I’d estimate somewhere around 1 in 30000 packets may be affected, given average use)
Anyway, a patch has been made, the dswifi CVS repository at devkitpro has been updated with the modified code, if you would like to update it. I will not be releasing an updated compiled release package for this bug, as there is a rewrite planned of the whole thing in the near future.
Sorry for the lack of updates lately, I’m still not quite sure what things I should be working on and what order to get them done in – I will post more when I finish making up my mind :)

Another revision hits the presses, get your downloads at the Sourceforge dswifi files page.
There are some major bugfixes in this release, but besides that, it’s not really changing functionality much.
Many of you might thank me for the following changes:
*Fixed major bug in arm7 (potentially causing arm9 to lose track of the buffer, and lock up I/O – odd that I hadn’t seen this happen, but someone else certainly had, and let me know about it)
*Changed default behavior to use internal allocation system for memory (fixes interrupt allocation bug) (this was pretty stupid on my part, but it is fixed now. Now, for default behavior, the lib allocates a block of memory at startup and the default functions alloc/free from this small heap (you can still use your own, if you like, but you have to add an extra flag to the arm9′s init call, which is WIFIINIT_OPTION_USECUSTOMALLOC. The default heap size is 128k, but there are flags to specify 64k, 256k, and 512k as alternate heap sizes. 128k is probably more than most people will need, if you need lots of TCP connections (>7), then you will need to specify a larger heap size.)
*Fixed bug that could cause _CANNOTCONNECT to come up incorrectly when associating (this was a “transient” problem, occurred due to 2 state changes that were offset by a bit, causing a condition I didn’t initially expect – it is fixed now and Wifi_AssocStatus() should return values consistent with what you’d expect)
*Fixed blocking send() to actually be blocking. (odd that, I evidently didn’t set it up earlier. It is blocking-capable now though)

I’m not re-releasing the examples package because they’re essentially the same; there is only one difference and that’s I’ve removed the sgIP_malloc and sgIP_free functions from the wifi_lib_test project.
On a side note, if you do notice any bugs in the lib, please do contact me, via IRC (#dswifi on efnet or blitzed) or other means. I’m expecting to set up a bugtracker in the near future too, so there will be a more formal way to yell at me :)
Ok, And for those of you who are unhappy cause your favorite feature got left out (be it probe requests, the WFC data alternate WEP key format, or native Nintendo USB key compatibility) – sorry, but this project has become stagnant, probably because I’m continually trying to refine it and just don’t feel like it anymore really; I’ve decided to put the whole lib on standby for the immediate future, and when I continue (probably mid-november) I’ll start with a complete rewrite of the 802.11 system and the TCP stack, for version 0.5 – at that point I’ll have the chance to actually do some of these things *right*, and I perhaps won’t be as frustrated with a codebase based on flawed assumptions and a general lack of understanding ;)

I’ll post again sometime soon with information about what projects I am presently going to be pursuing, and which ones are on the back burner.

Just because I figured this should probably make it out to the public; I’ve done something horrible in the wifi lib that could potentially make code running alongside it crash. To be clear, this will be fixed in 0.3b, which I am trying to get out as soon as I have time to do so.
However, just because it’s good to know if you’re developing an app using 0.3a, here’s the details of the bug:
In the wifi lib, when receiving packets and under a few other circumstances, the Wifi lib in it’s present form performs a malloc call from within the Wifi_Timer() call, and can also perform a malloc call from within the Wifi_Sync function, for FIFO messages – the only problem is both of those calls are often within interrupts, so the possibility of pre-empting a malloc in “normal” code (which, malloc is also called in printf, for reference) is nontrivial. The only problem with this, is that malloc wasn’t really designed for that, so a malloc call preempting another malloc call can cause fatal problems, corrupt the heap, or crash your code.
There are a few ways to deal with this problem, the first of which is to use a seperate heap allocator, which is the solution I’ll be incorperating into 0.3b. Those of you in the know can also implement some form of locking, wrapping allocation calls in interrupt disabling code, or something similar.
Another thing to make note of is this modification will cause the wifi lib to eat up a specific amount of memory on startup (which will be user selectable, probably set to 64k or 128k for a default) – there will also be the option to disable the new memory management if you have your own solution; which will be an extra flag that will be necessary to use in the init call!
So, if you’ve had some problems with the wifi lib crashing for the past few versions, it’s likely my fault, combined with your use of malloc/printf; all will be fixed soon though, and 0.3b includes a few other bugfixes for various other problems – I’ll list the fixes when it comes out.