Web Design Discussion > General Web Design Discussion
Setting up an instance of TM's DB and web front end
yakra:
Don't know if there's an existing topic I'm better off merging this into, but...
I've been following the instructions in Web/README.md and setting up a TM mirror on a CentOS system. There are some kinks to iron out; I'm still learning as I go.
A few observations & questions:
• From Web/README.md:
--- Quote ---This [lib/tm.conf] file needs to be readable by the web server but should not be served by the web server. Configure Apache to ensure this.
--- End quote ---
How do I do this exactly? I tried something in httpd.conf, and I believe I even restarted httpd; no luck.
• Updates on the updates page are still sorted by date, but in a different order within: the two CZE II603 entries from 2019-06-24 are swapped, as are the two WV WV817 entries from 2019-04-21. Appears deterministic on each site; reloading does not change it. On each server, one pair of entries is kept in CSV order, and the other swapped. Not a deal-breaker by any means, just something odd I noticed.
• Forum links: lib/tmheader.php links to "/forum", while index.php links to "http://forum.travelmapping.net"
• From devel/tools.php contains a hard link to "http://travelmapping.net/wptedit/", rather than a link relative to the DocumentRoot directory.
• graphs/index.php is broken -- below the "Filter by graph type" dropdown, I get:
--- Quote --- \n"; mysqli_report(MYSQLI_REPORT_STRICT); try { $tmdb = new mysqli($tmdbhost, $tmdbuser, $tmdbpasswd, $tmdbname); } catch ( Exception $e ) { //echoecho "
Failed to connect to database ".$tmdbname." on ".$tmdbhost." Please try again later.
"; exit; } $result = $tmdb->query("SELECT * FROM graphs ORDER BY descr, format"); $counter = 0; $prevRow; while ($counter < $result->num_rows) { // get three entries: collapsed, simple, then traveled for each $crow = $result->fetch_assoc(); $srow = $result->fetch_assoc(); $trow = $result->fetch_assoc(); $counter += 3; if ($crow == NULL || $srow == NULL || $trow == NULL) { // should produce some kind of error message continue; } // build table row (was: class=collapsed) echo "\n"; echo "\n"; echo "\n"; } $values = array(); $descr = array(); $result = $tmdb->query("SELECT * FROM graphTypes"); while ($row = $result->fetch_array()) { array_push($values, $row[0]); array_push($descr, $row[1]); } ?>
--- End quote ---
I'm still too new to sql & php, and how to configure it for TM, to know what's going on here. :)
• Other than that, the DB seems to be working. I was able to query a few of the tables, and what I saw made sense. stat.php, user/index.php, user/mapview.php, user/region.php, and user/system.php all look good.
• README.md does not cover what we need to do to get leaflet working. I was flying blind but tried to fly fast, and didn't do much research.
Initially, none of the maps worked. Then, just to make sure I have everything in place, I copied the /leaflet/ directory from noreaster. I don't know if there's anything specific to Jim's instance of leaflet I shouldn't be copying, any keys/tokens or anything (nothing immediately obvious from the contents); Jim, let me know if I shouldn't be doing that! FWIW, this isn't a public server; it's only visible to a few whitelisted IPs.
In any case, where's the proper place to find these files for someone setting up an instance of a TM server?
After copying /leaflet/, I got the aforementioned maps to work. Those in the HB still don't. Additionally copying the /leaflet-1.5.1/ directory didn't help.
Not sure what's going on here; my first thought is that it may be related to the recent Leaflet version upgrade, and maybe some changes to the production server haven't made it back to the Web repo yet. I'll nose around lab2 & noreaster, have a look at various .php & .js files, and see if any filesizes differ...
• Ingesting the .sql file took a bit over an hour on my machine. I don't understand why; my CPU use meters were all barely above 0% during the process. If disk access were the bottleneck here, it should have only taken ~20s to read the file... Jim, is this also your experience?
It rather dwarfs the time spent running the siteupdate progam itself. I wonder whether playing around with these values here would have an effect on performance...
yakra:
/lib:
FileSize in Web repoSize on noreasterjquery.tablesorter.pager.js(not present)4077tmjsfuncs.js3605736061tmphpfuncs.php1539015408
Edit: I see that these just refer to /leaflet-1.5.1 instead of /leaflet. No changes to maps; HB is still broken.
Forgot to mention, all I see on the HB is the page header. Not only is there no map; there's no Route Stats or Waypoints tables either.
Is shieldgen.php in the Web repo? If so, where do I find it?
After copying this from noreaster, I got the HB to work on lab2. WOOT!
I'd also like to understand the various tm.conf, tm.conf.standard, and tm.conf.updating in /lib/.
Edit: OK, I see we have this & this in localupdate.sh. Got it. How do they differ?
Schmedit: Let me guess:
--- Code: ---TravelMapping
travmap
[password]
localhost
--- End code ---
vs
--- Code: ---TravelMappingCopy
travmap
[password]
localhost
--- End code ---
?
yakra:
--- Quote from: yakra on June 26, 2019, 07:46:05 pm ---Ingesting the .sql file took a bit over an hour on my machine. I don't understand why; my CPU use meters were all barely above 0% during the process. If disk access were the bottleneck here, it should have only taken ~20s to read the file... Jim, is this also your experience?
It rather dwarfs the time spent running the siteupdate progam itself. I wonder whether playing around with these values here would have an effect on performance...
--- End quote ---
https://stackoverflow.com/questions/29643714/improve-speed-of-mysql-import
https://www.percona.com/blog/2014/01/28/10-mysql-performance-tuning-settings-after-installation/
https://dba.stackexchange.com/questions/13446/slow-load-speed-of-data-from-mysqldump
https://medium.com/@benmorel/high-speed-inserts-with-mysql-9d3dcd76f723
way out in the future or never at all
Setting innodb_flush_log_at_trx_commit=0 helped big time. Down to 23m13s to import TravelMapping.sql
Jim:
Thanks for all of these. I think it's very good to ensure we can install on another server and keep the documentation and code updated to facilitate this. I'm making some fixes and will add some things to the README.md.
Jim:
The DB takes under 5 minutes to ingest on noreaster. I know it's a pretty high-end server with lots of memory but I'm amazed to see such a difference on yours. Looks like I should also try those DB options and might be able to get site update time down significantly.
Navigation
[0] Message Index
[#] Next page
Go to full version