Unusual new behavior for region.php

[mapcat]

The table is responding strangely when multiple users have the same distance traveled.

https://travelmapping.net/user/region.php?units=miles&u=mapcat&rg=dc

In the DC region, 17 users have clinched everything. The table "Travelers in Region dc" shows two users ranked 1, one ranked 3, and varying numbers of other users ranked 4, 5, 7, 10, 13, and 16. All of these users have 47.64 miles in the region and until recently all of them were ranked 1.

It's not just happening with users at the top of the rankings. In the same table, the two users ranked 33 and the one ranked 35 have traveled the same routes. And it's not just affecting DC.

What changed?

[Markkos1992]

I just checked DE, and it has the same issue.  (https://travelmapping.net/user/region.php?u=markkos1992&rg=DE&units=miles)

[michih]

Yep, it is a general issue. LIE is a very small region. It might help on debugging: https://travelmapping.net/user/region.php?rg=lie

[yakra]

I suppose it's possible this could potentially be a data processing issue. Mapcat, how recently were they all ranked number one? Did you notice this after December 28th?

[yakra]

I'll update lab 2 with the same commits when I get home and check the db.
It could be a rounding issue; if it is data processing I have a good idea of what's probably causing it.

I'll load up both the C++ and python flavors and see if the issue happens with both.

[Jim]

Yes, looking like rounding before populating the DB:

Code: [Select]

> select * from clinchedoverallmileagebyregion where region='LIE' order by activemileage;
+--------+----------------+--------------------+----------------------+
| region | traveler       | activeMileage      | activePreviewMileage |
+--------+----------------+--------------------+----------------------+
| LIE    | Aseluit        | 0.9841289652476017 |   0.9841289652476017 |
| LIE    | panda80        | 1.1540176816881598 |   1.1540176816881598 |
| LIE    | charliezeb     |  1.742166048369596 |    1.742166048369596 |
| LIE    | M3200          | 2.0687608407804086 |   2.0687608407804086 |
| LIE    | dantheman      |  4.626547316794324 |    4.626547316794324 |
| LIE    | spinoza        | 7.1963766741696755 |   7.1963766741696755 |
| LIE    | griffith       |  7.196376674169677 |    7.196376674169677 |
| LIE    | martin0102     |  7.196376674169677 |    7.196376674169677 |
| LIE    | sneezy         |  7.196376674169677 |    7.196376674169677 |
| LIE    | the_spui_ninja |   7.94667527598344 |     7.94667527598344 |
| LIE    | bickendan      |  8.180505639417278 |    8.180505639417278 |
| LIE    | cougar1989     |  8.180505639417278 |    8.180505639417278 |
| LIE    | dave1693       |   8.46395000128351 |     8.46395000128351 |
| LIE    | philimon       |   8.46395000128351 |     8.46395000128351 |
| LIE    | niels          |  9.100692957671601 |    9.100692957671601 |
| LIE    | bartpetat      |  9.846927579160567 |    9.846927579160567 |
| LIE    | cinx           | 13.727240274465924 |   13.727240274465924 |
| LIE    | michih         | 13.727240274465927 |   13.727240274465927 |
+--------+----------------+--------------------+----------------------+


[yakra]

OK.  there's this one part of the code that's a bit of a kluge; logs look okay (since they're rounded to two decimal places) but I've always felt kind of nervous about it. For good reason as it turns out; I'd failed to take the database into account.
I do have an idea for a fix (and I believe there's a GitHub issue already open as well); should be easy to do but may take some unexpected tinkering.
Your choice I guess whether to revert to Python in the interim.

[Jim]

I'm willing to revert to Python for a bit if people find it enough of an annoyance.  Definitely not a critical error, as best I can tell it only affects rankings not the actual mileages (at least not anywhere near the precisions we display).

[mapcat]

I suppose it's possible this could potentially be a data processing issue. Mapcat, how recently were they all ranked number one? Did you notice this after December 28th?

I did not notice this on January 1. After that, I haven't paid attention until today.

[yakra]

system.php also dispays the same behavior mapcat describes in the OP.
https://travelmapping.net/user/system.php?u=sys=usadc
https://travelmapping.net/user/system.php?u=sys=usanh
https://travelmapping.net/user/system.php?u=rg=NH&sys=usame

[yakra]

This is the post where I mumble to myself
about what variables are at play, how they're populated, what's iterated & how, yadda yadda...

[yakra]

Jim, I believe this may be a problem with only multithreaded siteupdate, and not siteupdateST.
ISTR the new commonupdate uses siteupdateST for -t 1, yes?
Let's give that a try for tonight's update.


Edit: grepping thru archived sql files.
TravelMapping-2022-11-29@21:17:10.sql would be the first update done with C++, using siteupdateST I believe. There are 4 travelers with exactly 7.1963766741696755 mi.
Schmedit: Oops wrong month. I wanted December.
When was the first multithreaded siteupdate performed, as opposed to using siteupdateST?

[Jim]

Just finishing up a Python update for tonight.  I can run ST tomorrow if you'd like, or re-run tonight's if it's helpful to see it sooner.

[Markkos1992]

DE and DC are fixed now.

[yakra]

Python is fine, thanks.
I've got a good handle on what the problem is, and will post soon to describe it.
I've a basic idea on what to do for a fix, but implementation may get a little tricky if I want to avoid data races and also maintain good performance. In particular I'm thinking of the "messy dictionary of dictionaries" (or messy unordered_map of unordered_maps in C++ese).