Connected Route Question

[Markkos1992]

This is something that has confused me since we made it doable to see entire Connected Routes.

The Connected Route page for DE/PA/NJ I-295  currently has 139 drivers with 129 having clinched it.

However, when you look at the specific state pages, you get this:

DE: 110 drivers, 109 clinched (noting that this is one of if not the most traveled sections of route in TM overall)
NJ:  110 drivers, 57 clinched
PA:  65 drivers, 55 clinched

So theoretically, there should be at most 55 people (including myself) that have clinched the entire route.  I wonder what is in the program causing these odd discrepancies.

[Jim]

Doesn't sound right for sure.

I don't have time to check it out immediately, so I've created a GitHub Issue to make sure it does get investigated.

https://github.com/TravelMapping/Web/issues/542

[yakra]

HighwayData @ 4001f70ba7d298b2a019114696a0fce72455f76b
UserData @ bafd52681fb7ac64062a1aa7e563d351f543261a
DataProcessing @ 11be967a819fcd28f05582cb1050a721494c3596
Start: 2020-09-23 21:49:20.690292

The Connected Route page for DE/PA/NJ I-295  currently has 139 drivers with 129 having clinched it.
However, when you look at the specific state pages, you get this:
DE: 110 drivers, 109 clinched
NJ:  110 drivers, 57 clinched
PA:  65 drivers, 55 clinched

The HB still reports this as of this writing, on both noreaster & lab2.

user logs:
ConnectedRoute has 160 drivers; 42 clinched

Code: [Select]

[yakra@noreaster /home/www/tm/logs/users]$ grep 'Delaware - New Jersey - Pennsylvania' *.log | wc -l
     160
[yakra@noreaster /home/www/tm/logs/users]$ grep 'Delaware - New Jersey - Pennsylvania.*(100\.00%)$' *.log | wc -l
      42
Choped routes:

Code: [Select]

[yakra@noreaster /home/www/tm/logs/users]$ grep 'DE I-295:' *.log | wc -l
     144
[yakra@noreaster /home/www/tm/logs/users]$ grep 'DE I-295:.*(100\.00%)$' *.log | wc -l
     138
[yakra@noreaster /home/www/tm/logs/users]$ grep 'NJ I-295:' *.log | wc -l
     152
[yakra@noreaster /home/www/tm/logs/users]$ grep 'NJ I-295:.*(100\.00%)$' *.log | wc -l
      57
[yakra@noreaster /home/www/tm/logs/users]$ grep 'PA I-295:' *.log | wc -l
      65
[yakra@noreaster /home/www/tm/logs/users]$ grep 'PA I-295:.*(100\.00%)$' *.log | wc -l
      55Same results on lab2.

db queries on lab2:

Code: [Select]

mysql> select * from clinchedConnectedRoutes where route = 'de.i295';
160 rows in set (0.01 sec)
mysql> select * from clinchedConnectedRoutes where route = 'de.i295' and clinched = '1';
42 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'de.i295';
144 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'de.i295' and clinched = '1';
138 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'nj.i295';
152 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'nj.i295' and clinched = '1';
57 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'pa.i295';
65 rows in set (0.00 sec)
mysql> select * from clinchedRoutes where route = 'pa.i295' and clinched = '1';
55 rows in set (0.00 sec)...whether a C++ or Python-flavored siteupdate.

It appears something goes wrong on the Web (PHP?) side while/after getting results from the DB.

[yakra]

If I had some way to conveniently copy the tooltip text for Total Drivers & Total Clinched, maybe that could provide an avenue to explore...

[Jim]

Offending query seems likely to be here:

https://github.com/TravelMapping/Web/blob/02861e7723d7bc64f937fd59277359677c4e94c5/lib/getRouteStats.php#L51

[yakra]

I just ran that query on lab2 for de.i295, nj.i295 and pa.i295.
Results were 144, 138; 152, 57; 65, 55.
Are you getting different results on noreaster?

How about moving this topic to the public side of the forum, to get more sets of eyes on it? There was that time mikandkristie figured out some SQL issues...

[yakra]

HighwayData @ 9c66c8418df9657c20b1f1a99f82bb30cebcb4d6
UserData @ 09c99fc211a14ab8fae2dcf4e35521428b3c00d0

DataProcessing @ a905d2a / 3ac929e, same thing.
noreaster (MySQL 5.7?) shows 185 Total Drivers on US23; lab2 (MySQL 8.0) shows 186.

[Markkos1992]

Looking at MD I-95 this morning, I saw the following:

Total drivers: 109
Total clinched: 107 (98.17%)
Average traveled: 86.20 miles (79.15%)

Expanding that to all of I-95 has 202 out of 208 drivers having clinched.  I wonder if there is more going on than I originally suspected.  While I would expect many travelers to clinch I-95 in MD, I do not see how only two drivers would not have.

[mapmikey]

I know nothing about the programming that goes into TM but I can offer this which might be helpful in diagnosing:

If you pull up each individual state for I-5, you can hover the cursor at the lower part of "total clinched" and see the actual usernames that have fully clinched.  I did this for CA, OR and WA and wrote down the usernames.  Then I looked at the connected I-5 screen and it shows 51 clinching all of I-5.  When you hover over the "total clinched" on that screen, you get the clinched names from CA, followed by the clinchers of OR who were not on the CA list, followed by the WA clinchers who were not on either CA or OR.  In other words, the number of people who have clinched at least one full I-5 segment are given full clinch credit in the count.

Each individual I-5 page does appear to have the number of clinchers correct in that state.

Whatever causes that is likely doing a similar thing with distances.

[Markkos1992]

Looking at MD I-95 this morning, I saw the following:

Total drivers: 109
Total clinched: 107 (98.17%)
Average traveled: 86.20 miles (79.15%)

Expanding that to all of I-95 has 202 out of 208 drivers having clinched.  I wonder if there is more going on than I originally suspected.  While I would expect many travelers to clinch I-95 in MD, I do not see how only two drivers would not have.

I created a spreadsheet that I put in as a Google Doc to expand on this.

Searching "MD I-95" in Github got me 186 travelers in MD on I-95 with 105 having clinched it.  I know that this may not be exact due to some files using the multi-region lines now, but it should give a good idea of how much the numbers are off.

[yakra]

Searching "MD I-95" in Github got me 186 travelers in MD on I-95 with 105 having clinched it.  I know that this may not be exact due to some files using the multi-region lines now, but it should give a good idea of how much the numbers are off.

Code: [Select]

[yakra@noreaster /home/www/tm/logs/users]$ grep '  MD I-95: .* mi (.*)' * | wc -l
     189
[yakra@noreaster /home/www/tm/logs/users]$ grep '  MD I-95: .* mi (100.00%)' * | wc -l
     107


[Markkos1992]

yakra, is there a way to find out which two list files are being counted in the "109" number? I think it is clear that the 107 number having clinched it is correct.

[Jim]

Thanks for the reminders on this.  I'm taking a look now.  I'm currently not suspecting the SQL queries, but how the JS code uses the results to populate those stats.

[Jim]

The MD I-95 case is fixed with a mysqld parameter, as the results of the query to get the lists of drivers/clinchers were being truncated to 1024 characters.  No TM code changes for that.

[Jim]

I know nothing about the programming that goes into TM but I can offer this which might be helpful in diagnosing:

If you pull up each individual state for I-5, you can hover the cursor at the lower part of "total clinched" and see the actual usernames that have fully clinched.  I did this for CA, OR and WA and wrote down the usernames.  Then I looked at the connected I-5 screen and it shows 51 clinching all of I-5.  When you hover over the "total clinched" on that screen, you get the clinched names from CA, followed by the clinchers of OR who were not on the CA list, followed by the WA clinchers who were not on either CA or OR.  In other words, the number of people who have clinched at least one full I-5 segment are given full clinch credit in the count.

Each individual I-5 page does appear to have the number of clinchers correct in that state.

Whatever causes that is likely doing a similar thing with distances.

Good catch - investigating and should be fixed soon.