Friday, January 8, 2016

Practical CRTP (or lack thereof)

I have recently performed an experiment where I took the Loginator code and removed all dynamic polymorphism in place of static polymorphism. The motivation was the fact that since the sensors were all hard-wired to the board, all references to them could be figured at compile-time, and static polymorphism plus the optimizer and inliner could make "better" code. I also had an irrational fear of the vtables being corrupted.

Having gone through this exercise, I think I can say that CRTP is not the right way to do static polymorphism. It works, but it is ugly and infectious. If a class uses an object which is defined with CRTP, that class itself must also be made to use CRTP. I don't know if the machine code produced is faster, but I do know that it is significantly larger.

The motivation for changing back is the hardware description block which I am adding to the Loginator code, in hopes of being able to unify the Logomatic, Loginator, Rocketometer, Rollercoasterometer, and Pocketometer. My idea is to be able to load the same binary image into any LPC2148 based hardware I have or make, and have the software read the hardware description block to see what hardware it is attached to. This is all done at runtime, so we need dynamic polymorphism again.

CRTP isn't all bad. It works fine for instance in the dump class, where you can use it to make specializations of dump which are resolved at compile time. The problem arises when you want to pass a CRTP object as a parameter. At that point, the reference or pointer to the object has to be CRTP also, which frequently causes the whole class to need to be CRTP.

The idea then is that the general-purpose drivers are built into the code, and constructed at runtime from the hardware description block.

Tuesday, August 18, 2015

Magnum Torch

A magnum torch is an Extra Utilities block which prevents the spawning of hostile mobs within a certain space around it. How? Like this:

 public static boolean isInRangeOfTorch(Entity entity)
    for (int[] coord : magnumTorchRegistry) {
      if ((coord[0] == entity.field_70170_p.field_73011_w.field_76574_g) &&
        (entity.field_70170_p.func_72899_e(coord[1], coord[2], coord[3])) && ((entity.field_70170_p.func_147438_o(coord[1], coord[2], coord[3]) instanceof IAntiMobTorch)))
        TileEntity tile = entity.field_70170_p.func_147438_o(coord[1], coord[2], coord[3]);
        double dx = tile.field_145851_c + 0.5F - entity.field_70165_t;
        double dy = tile.field_145848_d + 0.5F - entity.field_70163_u;
        double dz = tile.field_145849_e + 0.5F - entity.field_70161_v;
        if ((dx * dx + dz * dz) / ((IAntiMobTorch)tile).getHorizontalTorchRangeSquared() + dy * dy / ((IAntiMobTorch)tile).getVerticalTorchRangeSquared() <= 1.0D) {
          return true;
    return false;

This is a method which supports both the chandelier and magnum torch. Each of those has the getHorizontalTorchRangeSquared and getVerticalTorchRangeSquared methods. It doesn't matter how this method gets called, or what the obfuscated functions are. The important bit is that line which works with dx, dy, and dz. Those are obviously the distance from the torch to the entity in question (the mob trying to spawn). The formula looks like this:

\[\frac{\Delta x ^2+\Delta z^2}{h^2}+\frac{\Delta y^2}{v^2} \le 1 \]

Rearranging slightly, we see the formula for the surface and interior of an ellipsoid:

\[\frac{\Delta x ^2}{h^2}+\frac{\Delta z^2}{h^2}+\frac{\Delta y^2}{v^2} \le 1 \]

The semi-axes in the horizontal direction are both \(h\), while the semi-axis in the vertical direction is \(v\). Looking at the implementation of these blocks we see:

public float getHorizontalTorchRangeSquared()
    if ((func_145838_q() instanceof BlockMagnumTorch)) {
      return 16384.0F;
    if ((func_145838_q() instanceof BlockChandelier)) {
      return 256.0F;
    return -1.0F;
  public float getVerticalTorchRangeSquared()
    if ((func_145838_q() instanceof BlockMagnumTorch)) {
      return 1024.0F;
    if ((func_145838_q() instanceof BlockChandelier)) {
      return 256.0F;
    return -1.0F;

So, the magnum torch clears a spheroid around itself with an equatorial radius of 128 meters and a polar radius of 32 meters. Similarly the chandelier clears a sphere around itself of radius 16 meters.

Monday, August 3, 2015

Gem of the Once-In-A-While: Ragel FSM generator

Yukari 3 ran a hand-built state machine to parse NMEA sentences coming from the GPS. It worked, but it was hard to maintain. I went looking for a tool that was a better solution. I was looking for something like a C++ template library that implements a state machine by abusing the template processor. What I found was Ragel.

This a nice, simple, clean program for generating finite state machines and weaving them into your programs. It does so in a way which I vastly prefer over Lex. It's main drawback is that since it is not a template library, it does require an external program to handle. Well, so do several other parts of my system. I use Perl without embarassment, so I will use Ragel similarly.

Ragel is structured very similarly to Yacc, in that it creates a program which executes user-chosen actions when a certain sentence form is encountered. However, it only handles part 1 languages, IE regular languages.

Ragel takes as input a source file written primarily in C/C++ (other languages are supported, but I don't use them) but which is marked with certain Ragel blocks. These blocks are used to define the machine and actions. The actions are written in the main language (C/C++ as I use it) and Ragel builds code which executes the state machine on an input stream and executes your actions as they come up.

It demonstrates a couple of interesting ideas, one of which is that you can just make up a new language and mix it into your existing source code.

Certain of the Ragel commands indicate where the state machine's tables should be written, where the state machine code should go, etc. Interfacing to the scanner is pretty simple.

A couple of examples below the fold:

Saturday, July 18, 2015

Sputnik Planum

Putting on my junior geologist hat, what it looks like here is that the smooth material in the north is different from the rougher material in the south. The smooth material looks like it shrunk and cracked, with some of the plates sliding downhill and pushing up those hills you see in the east.

Although, it looks like the polygonal cracks extend into (under) the rough material. You can see what looks like a whole plate of rough material surrounded by hills. Is the rough stuff on top of the smooth stuff?

Tuesday, July 14, 2015


As I type, there is a signal flashing across the solar system at the speed of light. It was transmitted by the New Horizons spacecraft, and while it is encoded in phase shift keying, circular polarization, ones and zeros, CCSDS packets, it carries a simple message. It's just a playback of engineering data. But, it carries the following important information, translated into English:

  1. I am still functioning properly and have survived flyby
  2. I have executed the observation sequence to this point, have made this many observations and recorded this much data
  3. I am going to shut up now and get back to recording data
Alternatively there is a lack of signal flashing across the solar system at the speed of dark, carrying the unfortunate message that New Horizons did not survive flyby, and this is the best picture of Pluto we will get for decades:

Saturday, June 27, 2015

Splitting a git repository

I keep all my hobby code in a big git repository. The problem is that it is too big. I have something like 28GB in it, of which the vast majority is data, by which I mean results of data-collection processes, such as photos, rocketometer data, etc. It also includes data acquired from other sources, such as spice kernels, shape models, image maps, etc. It is making everything slow.

So, the solution is to split the repository. We split it into code, data, and docs.

Now, how do we split a repository? One option is to use the data we have to build new repositories. We go through the existing one commit-by-commit, generate a diff, filter that diff so that only the right files go into the new repository, then make a new commit in the new repository, making sure to keep the commit message, user information, and timestamp.

That seems hard. Also, it seems like someone should have already done that. So, I did some research to see if anyone else has done this. Mostly I am looking for people who are permanently removing files from a repository. My idea is to make three copies of the existing repository, then remove the files that don't belong using the methods described on the Internet.

In the course of my research, I found a program called BFG. This program does a full remove of a large set of files, in a way that is much quicker than git filter-branch. In order to do this, we need to remove the files from the head of the repository, so we do things like this:

  1. Make three copies of the original workspace with git clone --mirror . These are going to be the three new repositories, so call them code.git, data.git, and docs.git
  2. For each repository, check it out with git clone (no --mirror). This working copy is temporary. We will use the code repository as typical.
  3. Because of the way BFG works, we have to rename any folders which are called Data or Docs. BFG doesn't remove just /Data, but any folder called Data. As it turned out, there were some such files. They needed to be either renamed (to data and docs, note lower case), deleted, or moved.
  4. Remove the files from the working copy with git rm -r Data Docs .
  5. Move all the files and folders from code down one level, since the old code folder will be the new root: git mv code/* .; git rm code
  6. Commit and push the removal and move
  7. Now go back to the new mirrored repository code.git and run java -jar ~/bfg-1.12.3.jar --delete-folders Data and java -jar ~/bfg-1.12.3.jar --delete-folders Docs . This process runs very quickly.
  8. At this point, the repository is cleaned, you can't see any evidence of the files, but there are garbage objects which need to be removed to actually make the repository take less space. To do this, run git reflog expire --expire=now --all --verbose and git gc --prune=now --aggressive . Git garbage collection takes a long time and an immense amount of memory, more than the 8GiB my file server actually had. I had to run the gc over the network on my game rig, which has 32GiB.
  9. Now the repository is shrunk. We went from 28GB to 3GB for the code repository.

Friday, June 26, 2015

More on sensor lag

I have done further analysis of the practice runs, and also hooked up the PPS and recorded some data while fixed.

From the PPS data, the robot takes an average of 0.54s from the time that the PPS fires to the time that it finishes receiving and parsing the RMC sentence. This is true every 4 seconds out of 5, but every fifth second, the GPS sends a burst of GSA and GSV sentences, which apparently delays the RMC to an agonizing 0.96s.

During practice run 3, the log file YUKARI02.SDS was recorded when the robot ran into the barrel. During its run on the second leg, it reached a peak speed of 6.57kt, or over 354cm'/s. It had a sustained speed on the second leg of over 300cm'/s . This is around 3m/s.

So, the sentence is typically processed 0.54s or 150cm' delayed, but up to 0.96 or over 300cm' delayed. Plus, the fixes are uniformly randomly distributed relative to the actual time of waypoint passage, with the delay equally likely to be any time between 0 and 1 second. This is another 0-300cm' of delay, meaning that the robot will detect and react to the waypoint between 1.5m and 6m beyond. I pulled the waypoint back 6.6m when I did the short waypoint test.