Author Topic: Tableau  (Read 2898 times)

0 Members and 1 Guest are viewing this topic.

Offline jayhawkco

  • Full Member
  • ***
  • Posts: 125
  • Last Login:February 05, 2024, 01:16:11 pm
Tableau
« on: January 29, 2021, 12:37:54 am »
Hey Jim (and others),
    Not to reinvent the wheel, but have you thought about using the map functionality of Tableau in any way?  It seems a lot of things that are less user friendly about Travel Mapping (.list files and such) could be pretty easily eliminated.  A fairly simple form on a web page could adjust each user's database of logged routes.  I'm working on mocking up a fairly simple version for Colorado state highways just as a proof of concept, but was just curious if you all had looked into Tableau at all first.  The only thing I might foresee an issue with for what I'm creating is connected routes, but I have some ideas bouncing around my head to fix those too.

Chris

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2732
  • Last Login:Today at 10:27:44 am
Re: Tableau
« Reply #1 on: January 29, 2021, 09:22:38 am »
I am not familiar with it, so I can't really say how it might fit in.  Please share any further details and your proof of concept, though.  Maybe it would end up being a good fit for TM in some way.

I will mention the main reason user data is submitted via .list files and why I have no plans to allow users to modify their travels interactively.  The TM database is read-only for all web-facing code.  This eliminates many potential security problems, and given that I'm not a database or security expert and that we can't risk any kind of security breach given than TM is housed on a server owned by my employer, using their power, network, etc.

Another thing to consider: the site update code that reads the highway data and processes .list files builds data structures that do not remain in memory after the big .sql file that populates the TM DB gets created.  There are consistency issues with the DB (route concurrency detections, stats and rankings stored in the DB) and updates to traveled-format graphs that would need fundamental changes if we were to switch from the static data we have now to something that would update any highway or user data dynamically.

Offline jayhawkco

  • Full Member
  • ***
  • Posts: 125
  • Last Login:February 05, 2024, 01:16:11 pm
Re: Tableau
« Reply #2 on: January 29, 2021, 05:35:08 pm »
I am not familiar with it, so I can't really say how it might fit in.  Please share any further details and your proof of concept, though.  Maybe it would end up being a good fit for TM in some way.

I will mention the main reason user data is submitted via .list files and why I have no plans to allow users to modify their travels interactively.  The TM database is read-only for all web-facing code.  This eliminates many potential security problems, and given that I'm not a database or security expert and that we can't risk any kind of security breach given than TM is housed on a server owned by my employer, using their power, network, etc.

Another thing to consider: the site update code that reads the highway data and processes .list files builds data structures that do not remain in memory after the big .sql file that populates the TM DB gets created.  There are consistency issues with the DB (route concurrency detections, stats and rankings stored in the DB) and updates to traveled-format graphs that would need fundamental changes if we were to switch from the static data we have now to something that would update any highway or user data dynamically.

Basically Tableau is a data visualization software that can work with both static and dynamic data sources.  I kind of liken it to a PivotTable mixed with some SQL.  The particular perk that I think would be valuable for the site is that it can generate (automatically in many cases) maps on OpenStreetMaps. 


https://drive.google.com/file/d/1lTJk1sHGMqMfcmxoSJ_zDxR2A4d_ADQW/view?usp=sharing

https://drive.google.com/file/d/178AQavV5f5-3DxblCEd9DkKHKmG0xVbx/view?usp=sharing

(Tried to embed these as images but it doesn't seem to like me).

This is super bare bones, with just CO125 and CO127's waypoints added in.  Basically I just created an excel spreadsheet (although Tableau could read the .wpt files since it can also take in plain text and use delimiters to get the data it needs) that has a column for highway, the point order (i.e. Waypoint1, Waypoint2, etc.), the waypoint name, the latitude, the longitude, a formula measuring the distance between waypoints, and the a column which indicates if the traveler had been to that waypoint or not.  The program takes in that info and auto-generates the map with the portion listed as traveled highlighted in orange, and then the text "crosstab" chart as well.  Obviously I didn't put all of the info that your site has (highway system, etc.). 

Re: servers and security.  How big is the database that we're working with now?  Just curious if we would ever want to charge an incredibly minimal subscription fee ($1 a month or something) to pay for a couple of gigs of cloud server access.

Chris

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2732
  • Last Login:Today at 10:27:44 am
Re: Tableau
« Reply #3 on: January 29, 2021, 08:19:57 pm »
Re: servers and security.  How big is the database that we're working with now?  Just curious if we would ever want to charge an incredibly minimal subscription fee ($1 a month or something) to pay for a couple of gigs of cloud server access.

We have two copies so one is complete and live while the other is updating, each about 800MB.

If we were ever to move to a situation where the DB could be changed by web-facing code, a hosting solution might make more sense.