system.php: System Stats & mileage figures

[yakra]

crossposted to a github issue

http://tm.teresco.org/user/system.php?sys=usame&u=yakra
http://tm.teresco.org/user/system.php?sys=usame&u=deathtopumpkins

System Stats table:
• system.php lists my mileage on usame as 2756.24/4845.83
• deathtopumpkins has mileage on usame of 2600.10/4454.29

I looked at Oscar's stats, then at all 3 of us in usanh. Different total overall mileage for all 3 of us, in both systems. I'm sure it'd be the same for any other number of site users and highway systems.

The top line, above the map, under the "Show Markers" checkbox:
• Traveled mileage is slightly less (<0.1) than the number reported in the System Stats table in most cases.
• Total mileage is the same for all 3 users. That's a good sign, an indication this may be the (more) "correct" figure. It mismatches the figure in usame-all.csv -- probably because the figure in system.php doesn't account for multiplexes.

To investigate this further, using the data in my "Statistics per Route" table:
Sum Clinched Mileage for all routes = 2756.27. Same ballpark as 2756.17 & 2756.24. Cumulative rounding errors, yadda yadda.
Sum Total Mileage for all routes = 5651.06. Same ballpark as 5650.98. Cumulative rounding errors, yadda yadda.
So yes, this would confirm that multiplexes are not counted in the system.php figure.

But where & how did the figures in the System Stats table go wrong?

[Jim]

I can't investigate right now, but a question to start: do we know if this is new, or just a problem no one has noticed before?  No changes have been made to the data processing script in a long time, but there's a lot more data being processed than ever before with the huge expansions in Asia and Europe.  It's possible we've hit some problems that only show up with our now-larger data sets.

[yakra]

do we know if this is new, or just a problem no one has noticed before?

In my case, I can't say for certain either way.

[mapcat]

Not only are the totals not matching up, but the rankings are wonky. Example:

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

It says my rank is 24, but I checked http://tm.teresco.org/devlogs/usaif-all.csv and according to my total mileage, I should be ranked #3. User lkefct, who should be #2 based on total mileage, is ranked #14 on his usaif stats page, and oscar, who should be #1, has a rank of 11 on his stats page.

Note that since usaif-all has no total mileage column (it's broken down by state), I had to calculate totals myself. Total mileage for the system is 1204.92; oscar has 1188.41, lkefct has 1014.81, and I have 997.57.

[Jim]

Not only are the totals not matching up, but the rankings are wonky. Example:

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

It says my rank is 24, but I checked http://tm.teresco.org/devlogs/usaif-all.csv and according to my total mileage, I should be ranked #3. User lkefct, who should be #2 based on total mileage, is ranked #14 on his usaif stats page, and oscar, who should be #1, has a rank of 11 on his stats page.

Note that since usaif-all has no total mileage column (it's broken down by state), I had to calculate totals myself. Total mileage for the system is 1204.92; oscar has 1188.41, lkefct has 1014.81, and I have 997.57.

I'm going to tackle user/system.php and user/region.php next in my conversion to use the consolidated DB.  I have put a note into the code to remind me to check this out once the system.php page is using the new DB.

[Jim]

I'm thinking that rather than digging into the PHP code that calculates the ranks, I might augment the DB to have the ranking info.  It's  whole lot easier to calculate from my Python data structures during site update than to do in PHP.  At the cost of a small increase in DB size, we'll make page loads more efficient (since PHP won't be calculating everyone's stats, sorting them, and then searching for the correct user).  I'm going to put that aside until I can get the rest of the pages using the consolidated DB.

[Jim]

It was easy enough to fix things without further DB changes.  Check out

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

for various users and systems, and optionally restricted by region (with rg=) and let me know if you still notice problems.

[yakra]

It was easy enough to fix things without further DB changes.  Check out

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

for various users and systems, and optionally restricted by region (with rg=) and let me know if you still notice problems.

Compare system stats on:
http://tm.teresco.org/user/system.php?u=yakra&sys=usame
http://tm.teresco.org/user/system.php?u=deathtopumpkins&sys=usame

[Jim]

It was easy enough to fix things without further DB changes.  Check out

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

for various users and systems, and optionally restricted by region (with rg=) and let me know if you still notice problems.

Compare system stats on:
http://tm.teresco.org/user/system.php?u=yakra&sys=usame
http://tm.teresco.org/user/system.php?u=deathtopumpkins&sys=usame

I'll investigate, thanks.  The total mileage numbers should come from exactly the same DB entry in each case.  Obviously that's not happening.

[Jim]

It was easy enough to fix things without further DB changes.  Check out

http://tm.teresco.org/user/system.php?u=mapcat&sys=usaif

for various users and systems, and optionally restricted by region (with rg=) and let me know if you still notice problems.

Compare system stats on:
http://tm.teresco.org/user/system.php?u=yakra&sys=usame
http://tm.teresco.org/user/system.php?u=deathtopumpkins&sys=usame

No wait, we're OK.  That's the old version.  Still need to use tmtest.teresco.org.  I haven't put the new code into the primary tm.teresco.org area.  Sorry about that.

[yakra]

Oops. My bad. I knew about the different URLs, just forgot to check that I was pointed at the right place there.

[Jim]

Oops. My bad. I knew about the different URLs, just forgot to check that I was pointed at the right place there.

You just followed by URL from my post, which was perfectly reasonable.