Author Topic: Suggestions to Programmatically Audit .list File  (Read 24661 times)

0 Members and 1 Guest are viewing this topic.

Offline Hoss6884

  • Newbie
  • *
  • Posts: 15
  • Gender: Male
  • Last Login:August 01, 2022, 11:02:21 am
Suggestions to Programmatically Audit .list File
« on: December 28, 2019, 02:36:00 pm »
Inevitably, I sometimes come across errors or omissions when updating my list file (i.e. concurrencies I didn't know I had or didn't capture).  I'd like to do a mass audit of my .list file at some point to make sure I'm 100% accurate with segments and waypoint names, but before I invest the time, does anyone have any suggestions on how to do it programmatically?  I'm willing to dust off my coding skills, if necessary.

Offline michih

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4782
  • Last Login:Today at 01:08:33 pm
Re: Suggestions to Programmatically Audit .list File
« Reply #1 on: December 28, 2019, 02:43:42 pm »
I'm not sure, maybe I got you wrong..... Try to answer paritally...

i.e. concurrencies I didn't know I had or didn't capture

It is not necessary to had these because concurrencies are automatically detected and you are credited for it - if the concurrent routes are of different systems, it counts for all systems.

I'd like to do a mass audit of my .list file at some point to make sure I'm 100% accurate with segments and waypoint names

That you have valid waypoints? Don't you know your user log file: http://travelmapping.net/logs/users/hoss6884.log
Everything looks fine for you. An example with errors.

Offline Hoss6884

  • Newbie
  • *
  • Posts: 15
  • Gender: Male
  • Last Login:August 01, 2022, 11:02:21 am
Re: Suggestions to Programmatically Audit .list File
« Reply #2 on: December 28, 2019, 03:46:13 pm »
Thanks for the response.

I’m sorry that I should have been clearer with my original question.  I understand I get credit for concurrencies regardless whether they are included in my file, but I would still like to be able to capture them if I don’t have them listed.  Same if a waypoint name changes - I know I still get credit but would like to have the most updated waypoint listed.

I'm not sure, maybe I got you wrong..... Try to answer paritally...

i.e. concurrencies I didn't know I had or didn't capture

It is not necessary to had these because concurrencies are automatically detected and you are credited for it - if the concurrent routes are of different systems, it counts for all systems.

I'd like to do a mass audit of my .list file at some point to make sure I'm 100% accurate with segments and waypoint names

That you have valid waypoints? Don't you know your user log file: http://travelmapping.net/logs/users/hoss6884.log
Everything looks fine for you. An example with errors.

Offline michih

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4782
  • Last Login:Today at 01:08:33 pm
Re: Suggestions to Programmatically Audit .list File
« Reply #3 on: December 28, 2019, 05:01:42 pm »
I’m sorry that I should have been clearer with my original question.  I understand I get credit for concurrencies regardless whether they are included in my file, but I would still like to be able to capture them if I don’t have them listed.  Same if a waypoint name changes - I know I still get credit but would like to have the most updated waypoint listed.

Got it :) I would also be interested in getting rid off alt labels from my user list file. It's all processed during the site update which generates the DB for the production server.

There is  python (standard, used by site update) and cpp (optional) code doing it: https://github.com/TravelMapping/DataProcessing/tree/master/siteupdate

I think that it should be this. Jim has a server and users can get space there. It's possible to clone hwy data, user data and the data processing repo so that you can set up your own instance to run a site update. It's mostly used by hwy data managers to make sure that Jim's site update won't fail with their modifications. I also use it to check that my latest list file changes are correct. And I can compare the log file stats to the one on the site.

You could try to get familar with the site update code and modify it to your purpose. You could output a log file or directly modify your list file or whatever.
Only yakra and Jim are currently familar with the code if you have any (more) detailed question.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2846
  • Last Login:Today at 01:20:51 pm
Re: Suggestions to Programmatically Audit .list File
« Reply #4 on: December 28, 2019, 05:07:01 pm »
This is something that came up quite a while ago in an old thread:

http://forum.travelmapping.net/index.php?topic=1827.msg4443#msg4443

I would like to add this feature when time permits.  Opening a GitHub Issue.
« Last Edit: December 28, 2019, 05:10:04 pm by Jim »

Offline oscar

  • TM Collaborator
  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1576
  • Last Login:Today at 05:41:17 pm
    • Hot Springs and Highways pages
Re: Suggestions to Programmatically Audit .list File
« Reply #5 on: December 28, 2019, 06:07:03 pm »
The part that would be most useful for me is detection of list file lines that are completely concurrent with others. In early CHM days, concurrences were not automatically detected, and so we had to enter them in our list files. That bulks up my list file, though I have manually removed many of the duplicate lines during jurisdiction-by-jurisdiction list entry cleanups.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2846
  • Last Login:Today at 01:20:51 pm
Re: Suggestions to Programmatically Audit .list File
« Reply #6 on: December 28, 2019, 06:20:51 pm »
The part that would be most useful for me is detection of list file lines that are completely concurrent with others. In early CHM days, concurrences were not automatically detected, and so we had to enter them in our list files. That bulks up my list file, though I have manually removed many of the duplicate lines during jurisdiction-by-jurisdiction list entry cleanups.

Noted in the GitHub Issue.

Offline Hoss6884

  • Newbie
  • *
  • Posts: 15
  • Gender: Male
  • Last Login:August 01, 2022, 11:02:21 am
Re: Suggestions to Programmatically Audit .list File
« Reply #7 on: December 28, 2019, 08:28:07 pm »
Thanks, Jim!  I couldn't see the old thread you linked but read the GitHub issue.

This is something that came up quite a while ago in an old thread:

http://forum.travelmapping.net/index.php?topic=1827.msg4443#msg4443

I would like to add this feature when time permits.  Opening a GitHub Issue.

Online yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 4348
  • Last Login:Today at 05:55:57 pm
  • I like C++
Re: Suggestions to Programmatically Audit .list File
« Reply #8 on: May 16, 2020, 02:30:09 pm »
I understand I get credit for concurrencies regardless whether they are included in my file, but I would still like to be able to capture them if I don’t have them listed.
Is what you want to do get a list of concurrent segments that you didn't explicitly list in your .list file, but got credited for due to concurrency detection?
If you're a Unix or Mac person, you can download concurrencies.log, and from the command line in the directory where you save it, run:
grep 'augment for traveler hoss6884' concurrencies.log

concurrencies.log is mainly for development purposes, so the output won't be very concise, but at least it's something.
Sri Syadasti Syadavaktavya Syadasti Syannasti Syadasti Cavaktavyasca Syadasti Syannasti Syadavatavyasca Syadasti Syannasti Syadavaktavyasca

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2846
  • Last Login:Today at 01:20:51 pm
Re: Suggestions to Programmatically Audit .list File
« Reply #9 on: May 16, 2020, 02:39:14 pm »
I understand I get credit for concurrencies regardless whether they are included in my file, but I would still like to be able to capture them if I don’t have them listed.
Is what you want to do get a list of concurrent segments that you didn't explicitly list in your .list file, but got credited for due to concurrency detection?
If you're a Unix or Mac person, you can download concurrencies.log, and from the command line in the directory where you save it, run:
grep 'augment for traveler hoss6884' concurrencies.log

concurrencies.log is mainly for development purposes, so the output won't be very concise, but at least it's something.

Yes, that would do it.  It can get a little overwhelming for longer concurrencies, though, since it marks every segment individually.