Opencaching Community
Cachen auf opencaching.de => Geocaching-Tools => Thema gestartet von: tomaszg am 28. Januar 2020, 03:28:20
-
Hi,
I'm developing a tool to compute various statistics related to Opencaching activity and a cache map. It supports all OC sites which use OKAPI. The tool itself is accompanied by a simple webpage to visualize the results and you can check it out here: http://tomaszg.pl/geo/.
It is still work in progress and especially all international features were not extensively tested. Myself, I use only OC.pl so many things may break. Also I still intend to add new features, so all ideas are welcome.
If anyone is interested to contribute, the source code is hosted at https://gitlab.com/tomaszg/geostat.
-
Hi Tomasz, it seems that this year 2020 ist the year of statictics.
We have already two statictics for oc.de available ;)
Regarding Your website: My nick (mic@) is not accepted:
==> Incorrect characters in nick d.
-
Regarding Your website: My nick (mic@) is not accepted:
==> Incorrect characters in nick d.
Ah, didn't expect a "@". Should work now.
BTW. Your German language question captcha is a bit annoying ;)
-
Ah, didn't expect a "@". Should work now.
Wow, thx for the fast reaction.
The statistic is very beautiful !!!
Is there a chance to get an overall statistic, for example a heatmap of ALL oc.de users?
-
Is there a chance to get an overall statistic, for example a heatmap of ALL oc.de users?
Do you mean just a heatmap of cache locations or heatmap of finds? In any case OKAPI doesn't allow it easily. It limits the amount of data user can pull. I would have to clone the database on my end and work with it. I'm doing that for OC.pl to plot caches not found but it is slow and requires storing the data locally on my server. I could try to do it. For OC.pl it is easy to get the cache heatmap by checking "not found" toggle and specifying a user without any finds (or with minimal one).
BTW. I just realized that OC.de hosts a lot of "Safari" reverse caches. I should probably try to exclude them from geographic part of stats. Too bad they don't have a separate cache type... I explicitly excluded "own" and "mobile" type of caches already.
-
I downloaded the data dump and rendered a cache density heat map:
(http://tomaszg.pl/geo/geo_de.jpg)
(http://tomaszg.pl/geo/geo_de_eu.jpg)
-
Do you mean just a heatmap of cache locations or heatmap of finds?
Correct.
So the result would be something like this here:
http://www.flopp.net/heatmap
Unfortunately this heatmap is only for caches, not for finds.
BTW. I just realized that OC.de hosts a lot of "Safari" reverse caches. I should probably try to exclude them from geographic part of stats. Too bad they don't have a separate cache type... I explicitly excluded "own" and "mobile" type of caches already.
All safari caches have a specific attribute.
Maybe this could help for building the right filter.
And "maybe" we will fix this problem soon, because there is the idea of
simplifying the cache types (e.g. remove math caches and introduce safari cache).
Your maps look great - thank You very much!
-
Correct.
So the result would be something like this here:
http://www.flopp.net/heatmap
Unfortunately this heatmap is only for caches, not for finds.
If I find the time, I'll try to prepare something for finds. I'll probably only upload a generated image, but it might be fun to create a web interface to be able to trace geocaching activity in a given time period.
All safari caches have a specific attribute.
Maybe this could help for building the right filter.
And "maybe" we will fix this problem soon, because there is the idea of
simplifying the cache types (e.g. remove math caches and introduce safari cache).
I noticed that. My code is not yet ready to handle attributes but when it does I'll try to filter them out.
-
By the way: It seems like umlauts are not allowed inside the user name as well at the moment.
-
Now it should work. I managed to get better unicode support in my web script and I hope I didn't break anything else by doing that.
-
Added a bit of stuff last month: display of "trail" instead of heatmap (works so-so), time range (i.e. limiting stats to caches found in a certain period), added stars next to names of recommended caches, and a few minor polishing things.
-
Thank You for your great wirk.
I just trid to view my statictics again, but it failed:
OKAPI error: "Parameter 'cache_codes' has invalid value: Duplicate codes detected (make sure each cache is referenced only once)."
I tried it with "mic@" and german map
-
It seems I reintroduced an old bug with these changes. Should be fixed now.
-
Should be fixed now.
Yes, it works smoothly again. Thank You for Your work!
Normally I am not a big statistic fan, but it makes fun to watch my caching behaviour in numbers and caches.
Very cool!
-
Is it possible to allow statistics only for the owner himself/herself?
Actual I am possible to view any oc-user just by replacing USERNAME with the correct user (e.g.: zelda):
http://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=USERNAME&stamp=15&map=Germany
turns to
http://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=zelda&stamp=15&map=Germany
So it´s a bit like "big brother is watching you"...
-
Well, I can't do anything about it, it's Opencaching which is open and shares all the data with everybody :) I'm not asking anyone their OC password to identify them. It might be possible to devise some way to do it (not easily for me), but that wouldn't change the fact that all this data is for anyone to grab. Just entering somebody's OC profile you can see where and when he was in the past just by listing caches found. It even offers the possibility to export this data in easily parsable form, like gpx or kml. Simply dropping it into Google Maps would yield similar result.
Bottom line - it is something which could be resolved in a meaningful way only on OC end
-
I'm developing a tool to compute various statistics related to Opencaching activity and a cache map. It supports all OC sites which use OKAPI. The tool itself is accompanied by a simple webpage to visualize the results and you can check it out here: http://tomaszg.pl/geo/.
Is it possible to offer this tool on https too?
-
Sorry, my server doesn't support it. I never learned how to set it up properly. I'll look into it if I find the time :)
-
Sorry, my server doesn't support it.
Have You ever talked to oc.pl team? They use https 8)
https://www.opencaching.pl
Maybe they could inherit Your tool in their code?
-
In principle, everybody is free to deploy the tool wherever they wish. It is enough to grab the source and compile it.
I've been in contact with OCpl team. They are always looking for people to work on the service. It would be best just to incorporate it with the stats on the webpage. Regretfully that would require knowledge of PHP and I don't know enough to do it. My tool is written in C++.
I didn't consider the lack of https before to be relevant to the webpages I have. I just started the investigation and it might turn out to be quite simple. I'll need to contact IT support before though and if they reply quickly enough, I may have it working tomorrow.
-
Okay, it should work now with https.
-
Okay, it should work now with https.
Perfect 8)
One more thing:
When I go to https://tomaszg.pl/geo and want to start with myself (nick=mic@ / map=Germany)
then the result is:
https://tomaszg.pl/cgi-bin/geo-form.pl?nick_pl=&nick_de=mic%40&nick_us=&nick_nl=&nick_ro=&nick_uk=&stamp=15&start_date=&end_date=&map=Germany
But
https://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=mic%40&stamp=15&map=Germany
works the same...
So my question is: Could You erade useless information like &nick_nl=&
-
I would need to plug in some javascript to do it, as this is normal behaviour for HTML. I'm reluctant to use JS for such trivial thing, even more so since it is purely aesthetic :)
-
I would need to plug in some javascript to do it
I agree, no need for JS. My wish was only a "nice-to-have" feature for a smaller URL, but it works with long URL too.
THX again for all Your work, its brilliant.
Maybe You should call it Project OC 8)
Best wishes, Mic@
-
One minor bug. Please take a look at the Difficulty / terrain matrix:
https://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=zelda&stamp=15&map=Germany
The calculated values (Total 26 combinations found out of 81) are wrong.
-
Thanks for spotting! It was caused by simple copy/paste error, fixed now.
-
Okay, it should work now with https.
Have You stepped back to http?
Because http://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=mic%40&stamp=15&map=Germany works,
but https://tomaszg.pl/cgi-bin/geo-form.pl?nick_de=mic%40&stamp=15&map=Germany is not working anymore?!
-
I have some temporary configuration problems and I'm now working to fix it. Broke something last night, it seems.
Edit: I think everything is fixed now.
-
HeyHo !
Thangs tomaszg ! It a very cool project ! Stats for Opencaching !
Can and do you want to install a filter for all FTF hunters that counts the FTF ?
https://project-gc.com/ProfileStats/jodaempa#FTF
Great Thangs JoDaEmPa !
-
I wanted to implement some FTF stats, but regretfully it is rather difficult. I can't in a straightforward way check it via Okapi without parsing all log entries. I'm trying to find a way to do it in a reasonable way, but didn't figure it out yet.
FWIW, Project-GC checks for word "FTF" in the log to figure it out. I don't think many people put it in their logs (I know I don't usually) so it is not the best approach.
-
FWIW, Project-GC checks for word "FTF" in the log to figure it out. I don't think many people put it in their logs (I know I don't usually) so it is not the best approach.
As an alternative, Project-GC lets you use and configure a public Bookmark List for your FTFs, which is the method I use. I think it is more reliable than looking at log entries, as there may be other reasons to write about FTF in a log entry.
-
Right, but I'm not sure what would be an alternative of this approach for OC. I see you have some kind of lists on OCDE which could be used for this end. On the other hand, we don't have such a thing on OCPL as far as I know. I would prefer to have some solution that would work across the board... Uploading a file with a custom list of cache codes would be tedious.
-
At
project-gc.com also uses a filter with {FTF} entries!
But the method of j-he to put the {FTF's} in a bookmark list is probably the easiest method!
Thanks tomaszg
{translated with Google}
-
It supports all OC sites which use OKAPI.
Hi Tomasz, are you able to produce the OC statistic in a more vivid way (not picture, but video).
Here is what I mean: https://www.youtube.com/watch?v=kvhOL2AfJSo
-
It should be possible. Some kind of animated GIF or PNG probably would be easiest. I'll think about it. Not sure when I will have time to code it though.
-
There is now a basic support for animated maps. Not sure if that is exactly what you meant. Right now it is quite limited. Caches are displayed sequentially not taking into account time span between actual finds. I should probably also add some timestamp on the picture. There is also no way to control the playback of GIF at the moment, but I'm looking into shiny JS code that could make it more flexible.
Ah, it is also rather slow, even for ~500 caches. Be patient with the server ;)
-
I tried it, but no answer came (maybe I have too many caches found):
One important thing I forgot to mention (sorry):
This video should be for all caches on the platform (e.g. whole oc.de) and not for a specific user.
So this means that for our platform (oc.de) there are 25.313 caches to pop up in this video.
I presume that this is too much data for Your server...
-
You have about the same amount of caches as I do, so it should work. Maybe it was caused by Germany map being a bit bigger than Poland. However my app did produce the map, it took it only longer than HTTP server wanted to wait. It can be seen by a direct link: https://tomaszg.pl/geo/map_mic@.gif
As for the whole OCde - it can be done, but I would need to change the approach. Plotting a frame per cache would make the animation too long :D It should be day-based, I think. It would still take more time, but I could render it and put it on Youtube for example.
-
Plotting a frame per cache would make the animation too long
Please keep in mind that caches come (publish) and go (archive).
Right now we have 25000 active caches but this number must be multiplied.
I don´t know the exact factor, so just assume there have been 50000 caches in total.
When You plot 1000 caches per second from the beginning (13. August 2005) until today,
the video would last about 1-2 min, which is a good duration...
...and not too long like this one here :-)
https://www.youtube.com/watch?v=g2ZLxtqJuKE
-
I was thinking about plotting each day separately as a new frame. I didn't think about archived caches, my simple approach doesn't work well with erasing anything...