Author Topic: Unusual new behavior for region.php  (Read 30717 times)

0 Members and 2 Guests are viewing this topic.

Offline mapcat

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1761
  • Last Login:Yesterday at 09:49:04 pm
Unusual new behavior for region.php
« on: January 21, 2023, 11:37:42 am »
The table is responding strangely when multiple users have the same distance traveled.

https://travelmapping.net/user/region.php?units=miles&u=mapcat&rg=dc

In the DC region, 17 users have clinched everything. The table "Travelers in Region dc" shows two users ranked 1, one ranked 3, and varying numbers of other users ranked 4, 5, 7, 10, 13, and 16. All of these users have 47.64 miles in the region and until recently all of them were ranked 1.

It's not just happening with users at the top of the rankings. In the same table, the two users ranked 33 and the one ranked 35 have traveled the same routes. And it's not just affecting DC.

What changed?
Clinched:

Offline Markkos1992

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3291
  • Last Login:Today at 11:02:55 pm
Re: Unusual new behavior for region.php
« Reply #1 on: January 21, 2023, 12:58:54 pm »
I just checked DE, and it has the same issue.  (https://travelmapping.net/user/region.php?u=markkos1992&rg=DE&units=miles)

Offline michih

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4839
  • Last Login:Today at 11:59:34 am
Re: Unusual new behavior for region.php
« Reply #2 on: January 21, 2023, 02:32:57 pm »
Yep, it is a general issue. LIE is a very small region. It might help on debugging: https://travelmapping.net/user/region.php?rg=lie

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #3 on: January 21, 2023, 04:19:23 pm »
I suppose it's possible this could potentially be a data processing issue. Mapcat, how recently were they all ranked number one? Did you notice this after December 28th?
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #4 on: January 21, 2023, 05:22:48 pm »
I'll update lab 2 with the same commits when I get home and check the db.
It could be a rounding issue; if it is data processing I have a good idea of what's probably causing it.

I'll load up both the C++ and python flavors and see if the issue happens with both.
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2854
  • Last Login:Today at 10:22:46 pm
Re: Unusual new behavior for region.php
« Reply #5 on: January 21, 2023, 05:37:00 pm »
Yes, looking like rounding before populating the DB:

Code: [Select]
> select * from clinchedoverallmileagebyregion where region='LIE' order by activemileage;
+--------+----------------+--------------------+----------------------+
| region | traveler       | activeMileage      | activePreviewMileage |
+--------+----------------+--------------------+----------------------+
| LIE    | Aseluit        | 0.9841289652476017 |   0.9841289652476017 |
| LIE    | panda80        | 1.1540176816881598 |   1.1540176816881598 |
| LIE    | charliezeb     |  1.742166048369596 |    1.742166048369596 |
| LIE    | M3200          | 2.0687608407804086 |   2.0687608407804086 |
| LIE    | dantheman      |  4.626547316794324 |    4.626547316794324 |
| LIE    | spinoza        | 7.1963766741696755 |   7.1963766741696755 |
| LIE    | griffith       |  7.196376674169677 |    7.196376674169677 |
| LIE    | martin0102     |  7.196376674169677 |    7.196376674169677 |
| LIE    | sneezy         |  7.196376674169677 |    7.196376674169677 |
| LIE    | the_spui_ninja |   7.94667527598344 |     7.94667527598344 |
| LIE    | bickendan      |  8.180505639417278 |    8.180505639417278 |
| LIE    | cougar1989     |  8.180505639417278 |    8.180505639417278 |
| LIE    | dave1693       |   8.46395000128351 |     8.46395000128351 |
| LIE    | philimon       |   8.46395000128351 |     8.46395000128351 |
| LIE    | niels          |  9.100692957671601 |    9.100692957671601 |
| LIE    | bartpetat      |  9.846927579160567 |    9.846927579160567 |
| LIE    | cinx           | 13.727240274465924 |   13.727240274465924 |
| LIE    | michih         | 13.727240274465927 |   13.727240274465927 |
+--------+----------------+--------------------+----------------------+

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #6 on: January 21, 2023, 06:17:10 pm »
OK.  there's this one part of the code that's a bit of a kluge; logs look okay (since they're rounded to two decimal places) but I've always felt kind of nervous about it. For good reason as it turns out; I'd failed to take the database into account.
I do have an idea for a fix (and I believe there's a GitHub issue already open as well); should be easy to do but may take some unexpected tinkering.
Your choice I guess whether to revert to Python in the interim.
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2854
  • Last Login:Today at 10:22:46 pm
Re: Unusual new behavior for region.php
« Reply #7 on: January 21, 2023, 06:47:52 pm »
I'm willing to revert to Python for a bit if people find it enough of an annoyance.  Definitely not a critical error, as best I can tell it only affects rankings not the actual mileages (at least not anywhere near the precisions we display).

Offline mapcat

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1761
  • Last Login:Yesterday at 09:49:04 pm
Re: Unusual new behavior for region.php
« Reply #8 on: January 21, 2023, 07:17:15 pm »
I suppose it's possible this could potentially be a data processing issue. Mapcat, how recently were they all ranked number one? Did you notice this after December 28th?

I did not notice this on January 1. After that, I haven't paid attention until today.
Clinched:

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #10 on: January 21, 2023, 08:36:32 pm »
This is the post where I mumble to myself
about what variables are at play, how they're populated, what's iterated & how, yadda yadda...
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #11 on: January 21, 2023, 08:48:35 pm »
Jim, I believe this may be a problem with only multithreaded siteupdate, and not siteupdateST.
ISTR the new commonupdate uses siteupdateST for -t 1, yes?
Let's give that a try for tonight's update.



Edit: grepping thru archived sql files.
TravelMapping-2022-11-29@21:17:10.sql would be the first update done with C++, using siteupdateST I believe. There are 4 travelers with exactly 7.1963766741696755 mi.
Schmedit: Oops wrong month. I wanted December.
When was the first multithreaded siteupdate performed, as opposed to using siteupdateST?
« Last Edit: January 21, 2023, 10:07:59 pm by yakra »
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2854
  • Last Login:Today at 10:22:46 pm
Re: Unusual new behavior for region.php
« Reply #12 on: January 21, 2023, 09:48:20 pm »
Just finishing up a Python update for tonight.  I can run ST tomorrow if you'd like, or re-run tonight's if it's helpful to see it sooner.

Offline Markkos1992

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3291
  • Last Login:Today at 11:02:55 pm
Re: Unusual new behavior for region.php
« Reply #13 on: January 21, 2023, 09:59:57 pm »
DE and DC are fixed now.

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4421
  • Last Login:Today at 12:12:32 pm
  • I like C++
Re: Unusual new behavior for region.php
« Reply #14 on: January 21, 2023, 10:34:58 pm »
Python is fine, thanks.
I've got a good handle on what the problem is, and will post soon to describe it.
I've a basic idea on what to do for a fix, but implementation may get a little tricky if I want to avoid data races and also maintain good performance. In particular I'm thinking of the "messy dictionary of dictionaries" (or messy unordered_map of unordered_maps in C++ese).
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca