Opencaching.de

Opencaching International => General discussion => Thema gestartet von: wrygiel am 20. August 2011, 12:15:44

Titel: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 20. August 2011, 12:15:44
Hey!

I am a developer from Poland. I would like you to comment on a project
we've just started. Would you be willing to join it?

http://code.google.com/p/opencaching-api/

It's a pre-release BETA version. We have it already installed on our
production site ( http://opencaching.pl/okapi/ ), but there are a
couple of unresolved issues which should be fixed prior the release.

In next couple of days I will be adding some working examples on how
to use the API.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: solar22 am 20. August 2011, 12:29:47
Hello,

I'm solar22 - opencaching.de developer.
Currently i'm working on a REST api.
I see your api is actualy more finished as our.
Maybe i can use your construct for oc.de

You can see the status of my work here: http://gruessung-online.de/oc/doc.html

Greetings

solar22

Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 20. August 2011, 12:47:15
I'm glad we paired up then!

I would be REALLY very very happy if OKAPI would be included in opencaching.de. I will gladly help you with the integration.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 20. August 2011, 12:49:24
Can I access your source code somewhere?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: solar22 am 20. August 2011, 13:22:25
Currently i develop on my local machine.
I have problems with the upload to github.com repo. (Access denied.)
I think, during the next week, i'm able to upload the first code.
Then i can post the link here.

Greetings.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 23. August 2011, 06:21:32
Wojtek add support for GPX and other GPS-compatible output formats.
http://code.google.com/p/opencaching-api/issues/detail?id=33
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 09:18:32
I did, just yesterday.
http://www.opencaching.pl/okapi/services/caches/formatters/gpx.html
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 23. August 2011, 09:38:58
Is this project (OKAPI) open for the public?
http://www.opencaching.pl/okapi/introduction.html

Should we advertize it a little bit?
Or do You want to keep it secret until it works?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 09:48:38
Sure, you may advertize it. ;)

(But method signatures are not yet fixated. I plan on doing this in a couple of days.)
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 10:14:06
(This is in response to a private message by oliver.)

> Hi wrygiel,

> i am Oliver, developer at oc.de. I am very happy to read your
> posting. The first impression is "wow".

Thanks very much :)

I'm glad to hear you liked it because one of my goals is to have OKAPI
working on both our sites (and later - possibly on every other
opencaching site).

> New oc.de code is based on smarty and has several incompatible
> changes and many improvements. However, some old beginner-stuff from
> my php-beginning time in 2003 is still present.

Currently, OKAPI does not use much of the "legacy code". In fact, the
only things I use are "sql" and "sqlValue" functions + some global
variables. So, the only important thing is the database structure.

I believe it is possible that the current version of OKAPI would work
with opencaching.de "as is", without any need for modifications, but I
would have to see your database to be sure. However, you may just try
it, installation is simple - just 5 minutes of work (see project main
page). It requires PHP 5.3.

In case of incompatibilities, this could be helped with some crude
conditions like "if ($GLOBALS['absolute_site_URI'] ==
'http://opencaching.de/') { sql1 } else { sql2 }".

If you choose to use OKAPI, I would expect for all code changes to be
commited into the main OKAPI repository. I DON'T want you to develop
your own OKAPI branch, I hope you understand why - the project is all
about compatibility!

In other words - I will be happy to accept all your changes to OKAPI,
*IF* exactly the same changes will go to opencaching.pl. And I really
hope you will agree on this - it would be the best option for both of
our sites.

> I have to view it in detail, but as i remember, the most important
> change for the API is the table "cache_status". Maybe oc.pl does not
> contain boolean columns allow_user_view allow_owner_edit_status
> allow_user_log . It is absolutly required to follow these settings -
> maybe waldek can add that columns to oc.pl code, too (if required).

I think I don't understand what you're saying. Why would API need a
change in table "cache_status"? Also, I don't know about these allow_*
columns. (I am new to OpenCaching project, I've first seen this
database only two months ago. I do not know much...)

> We at oc.de need to decide first what license to apply to the cache
> data. As i remember, oc.pl does CC licensing for cache data?

Our "Terms of Use" page is in BETA, as the rest of OKAPI. ;) In fact,
I just copied parts of the "opencaching.com API" licence, which I
found apropriate.

Do you think it is possible for us to agree on a common licence, for
both of our sites?

> Let me know if you need write permission to git repository
> (repository is public read). Or if you prefer to keep it in you own
> repository.

I want to keep OKAPI code separate. I think this will simplify
development and deployment process. The idea is to FIRST have things
changed in OKAPI repo, THEN merge them into all production sites.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 23. August 2011, 13:34:41
Moved the topic to the private subforum.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 23. August 2011, 13:51:44
Maybe it will be interested for test Oliver to installl OKAPI on OC DE and we can look how to optimize and tune or change to use this API to OC Nodes ?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 23. August 2011, 14:11:30
Hi wrygiel,

private messages are private. Else, i would create a topic in the public forum. So, for the future, it would be nice not to see my private messages in the forum or ask me before you publish it. No problem in this case and it will be no problem for most of my private messages at all.

However, you may just try
it, installation is simple - just 5 minutes of work (see project main
page). It requires PHP 5.3.

Okay, thats the first problem. Our linux distribution does not support PHP 5.3. Moving to newer distribution is still on Todo. If you cannot remove the PHP 5.3 depency, i will try to do the migration soon, but it will take some time to prepare everything.

If you choose to use OKAPI, I would expect for all code changes to be commited into the main OKAPI repository
(...)
In other words - I will be happy to accept all your changes to OKAPI,
*IF* exactly the same changes will go to opencaching.pl. And I really
hope you will agree on this - it would be the best option for both of
our sites.

I absolutly agree that there should be only one version and that this version should run on both sites without modification and customisation. However, as i have written, it is required to implement some modifications to meet the requirements of the new code. E.g. we have a cache status "locked, invisible". Not allowed to be viewed by users (only admins and owner), no logs allowed, status changes only allowed by admin. If the API does not follow that rules, we could get legal implications, because most times we set that status, there are problems with the copyright and we already had contact with a lawer who wants to see money from us, if we still publish the listing.

I think I don't understand what you're saying. Why would API need a change in table "cache_status"? Also, I don't know about these allow_* columns.

It is realy simple:

Each cache has a status-id (1, 2, 3, ...). When you want to know if a all users are allowed to view that cache listing, you only need to get the cache-status-id from table caches and read allow_user_view from table cache_status for that id. If it is 1, all users are allowed to view that cache. If it is 0, the user needs special permission - this may be because he is the owner of the listing or the user is admin.

It will be no problem to add these new database columns to the oc.pl databse. Then your API works again with both sites without modifications.

Zitat
Our "Terms of Use" page is in BETA, as the rest of OKAPI. ;) In fact,
I just copied parts of the "opencaching.com API" licence, which I
found apropriate.

Okay, then i have to check if the terms of use are affected by copyright laws. In germany, you cannot simply copy anything from anywhere. Especially terms of use are copyright protected, because the company has paid much money to get the terms of use from a lawer. It depends on the specific content ...

Zitat
Do you think it is possible for us to agree on a common licence, for
both of our sites?

I hope thats possible! We should commit at least for some sort of "opencaching network terms of use". Each node should support that terms of use. If a node wants to allow more than that, its okay. But a node should not restrict their thers of use more than the common OC network terms of use.

"OCN-TOU" = Opencaching Network - Terms Of Use

E.g. 1) If we decide not to allow commercial use in the "OCN-TOU", oc.pl may allow it for listings originating from oc.pl (but not for those from oc.de).

E.g. 2) If we decide to allow commercial use in the "OCN-TOU", oc.de may not forbid it.

It will be realy hard work to find "OCN-TOU" where all nodes aggree on. The more simply and clear they are, the better it is.


With kind regards,

Oliver
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 23. August 2011, 14:13:44
Maybe it will be interested for test Oliver to installl OKAPI on OC DE and we can look how to optimize and tune or change to use this API to OC Nodes ?

At the moment i am working on the migration of some oc.se-code (which has been migrated from oc.pl) to oc.de. There is a bunch of other work todo.

But solar22 could install the API on his local development machine and start with testing. He already did some coding on a oc.de-API ... with the OKAPI-project we can stop to work on our own API.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 19:25:07
> private messages are private. Else, i would create a topic in the
> public forum.

Okay, sorry about that. It seemed to me that our discussion would be
interesting for all other OC site-admins.

> Our linux distribution does not support PHP 5.3. Moving to newer
> distribution is still on Todo. If you cannot remove the PHP 5.3
> depency, i will try to do the migration soon, but it will take some
> time to prepare everything.

It would be hard. OKAPI is deeply dependant on namespaces - one of the
new features in PHP 5.3.

> I absolutly agree that there should be only one version and that
> this version should run on both sites without modification and
> customisation.

Great!

> However, as i have written, it is required to implement some
> modifications to meet the requirements of the new code. E.g. we have
> a cache status "locked, invisible". Not allowed to be viewed by
> users (only admins and owner), no logs allowed, status changes only
> allowed by admin.

Okay. There are two ways of dealing with that:

1. Oc.pl can incorporate that changes that you've made in oc.de. We
need Waldek's opinion on this - probably he is the one who would be
doing this. (I am responsible for OKAPI code only.)

2. We can make OKAPI work with both database structures with *the same
interface*. I am not 100% sure if this can be done, but it seems
possible.

To solar22: If I am not mistaken, in both cases, you'd be the one to
modify parts of the OKAPI code. I can give you commit rights, but
you'd have to have an account at Google.

Ad1. In this case you will modify OKAPI to work with oc.de, and then
we (you, Waldek an me) have to update oc.pl code with proper changes.

Ad2. In this case you will modify OKAPI to work both with oc.de and
oc.pl - you will be adding stuff like "if (I'm running on oc.pl) { do
womething } else { do something else }". (I'll call them "multisite
conditions" from now on.)

I think, the second approach would be easier for starters. Later, we
can get rid of these "mutli-site conditions", one by one.

What do you think? (Waldek? Oliver?)

> It will be no problem to add these new database columns to the oc.pl
> databse. Then your API works again with both sites without
> modifications.

If Waldek agrees to do this, then its fine with me! (In this case, I
will add support for this to OKAPI.) If not, then solar22 and I would
have to do more of "multi-site conditions".

> "OCN-TOU" = Opencaching Network - Terms Of Use

> E.g. 1) If we decide not to allow commercial use in the "OCN-TOU",
> oc.pl may allow it for listings originating from oc.pl (but not for
> those from oc.de).

> E.g. 2) If we decide to allow commercial use in the "OCN-TOU", oc.de
> may not forbid it.

> It will be realy hard work to find "OCN-TOU" where all nodes aggree
> on. The more simply and clear they are, the better it is.

Seems reasonable, but I'm really not good at legal stuff.

I tend to think that commercial is good, it powers the progress. Well,
I'm not 100% sure about that, but I think that, for example,
developers might be more keen to invest their time in an Android app,
if they had an option to "switch commercial" one day.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 20:47:20
BTW, I got some emails about developers wanting to start using OKAPI. I will have to start "fixating" methods soon. This is the proper time for you guys to add comments on method signatures and output formats. We won't be able to change these after people start using them. (We WILL be able to deprecate them and add new methods, but it's not the same - deprecated method will still have to be documented and will clutter the documentation pages.)

By "fixating" methods, I mean "acknowledging that they will be permanently backwards-compatible from now on".

For example, what do you think about "cache_wpt" parameter name (many methods require it), wouldn't it be "prettier" to call it "cachecode" or "ucid" (universal cache ID) or something? (I like such reference-entities to end with "id" or "code".)

Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 23. August 2011, 21:12:23
I just saw that Google Code has a nice "commenting" feature. While browsing repository (revisions or revision diffs), you may double click a line of code to add a comment, then click "publish your comments" to make it visible to other developers. Seems useful. http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/geocache.xml?r=59
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 23. August 2011, 22:38:53
Zitat von: wrygie
Sure, you may advertize it. ;)

OK, the posting is online
http://www.geoclub.de/viewtopic.php?f=127&t=57763
and here is some sniplet of comment, which You might find useful:

original:
...auch wenn ich bei der bbox statt SWNE doch WSEN bevorzugt hätte
sowie statt "|" ein schlichtes "," als trenner, da es dann identisch zur OSM xapi wäre.


translation:
...although I would have used WSEN instead of SWNE and
a simple "," instead of "|" as delimiter, so that it is identical to OSM xapi.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 23. August 2011, 23:06:37
For example, what do you think about "cache_wpt" parameter name (many methods require it), wouldn't it be "prettier" to call it "cachecode" or "ucid" (universal cache ID) or something? (I like such reference-entities to end with "id" or "code".)

Each cache has the following unique identifiers:
id - autoincrement id generated by mysql. Only valid for the specific node
uuid - GUID generated by mysql. There will only be one cache in the entire OC network with that id
wp_oc - waypoint within the OC network

I prefer this way (VB.NET):

Enum CacheIdType
  Id
  Uuid
  WpOc
End Enum

Class CacheId
  Public Id As String
  Public Type As CacheIdType

  Sub New(id As String, type As CacheIdType)
    Me.Id = id
    Me.Type = type
  End Sub
End Class

Function GetCacheInfo(ByVal id As CacheId) As CacheInfo

Client can simply use the function in this way:

info = Server->GetCacheInfo(new CacheId(4711, Uuid))
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 24. August 2011, 00:00:48
I think you misunderstood my question. I am only talking about a public NAME for the attribute, NOT THE SOURCE. The source is already decided - waypoint code.

Id is not unique, it should be kept hidden.
Uuid - I don't understand it's function. Who uses it and why? It also should be kept hidden.
waypoint - it's short and globally unique, the only valid option for a cache identifier. Opencaching.com uses it too.

Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 24. August 2011, 06:50:01

> However, as i have written, it is required to implement some
> modifications to meet the requirements of the new code. E.g. we have
> a cache status "locked, invisible". Not allowed to be viewed by
> users (only admins and owner), no logs allowed, status changes only
> allowed by admin.


In our cache_status we have

(1, 'Gotowa do szukania', 'Ready for search'),
(2, 'Tymczasowo niedostępna', 'Temporarily unavailable'),
(3, 'Zarchiwizowana', 'Archived'),
(4, 'Ukryta do czasu weryfikacji', 'Hidden by approvers to check'),
(5, 'Jeszcze niedostępna', 'Not yet available'),
(6, 'Zablokowana przez COG', 'Blocked by OC Team');

where status 4 is used to caches when are in query to approval by OC Team and nobody only owner and OC Team see this cache and discus with owner cache to make suggested changes to cache. Status 6 is status for blocked by OC Team and is cache is visible by owner and OC Tema but only OC Team can change status and after this owner can do more on own cache but any users can not see this cache
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 24. August 2011, 07:11:51
Id is not unique, it should be kept hidden.
Uuid - I don't understand it's function. Who uses it and why? It also should be kept hidden.
waypoint - it's short and globally unique, the only valid option for a cache identifier. Opencaching.com uses it too.

For caches,you are correct. There we may not need uuid. But we have other objects in the database like user, cache_logs, picuture. Each object can be identified by its uuid (universal unique id - its the ucid you suggested). When we use the uuid for other objects, it would be good to have this option for caches, too. Each node already has the uuid.

Waypoint may be initialized with NULL. If the code is not able to assign a waypoint with cache creation, the wp_oc is NULL and will be assigned later by a cronjob. This is a realy rare situation, but it may happen in error situations. Or it may happen when you create a cache, but publish it later. Also not, that the uuid of an object will never, never, never change. Its used for replication in OC network. The waypoint could be changed by the admin - maybe to honor a special user, he gets the waypoint OC1000 assigned manually. I never use primary keys with busniess meaning.

The local id of a cache was required as primary key, because doing sql-joins on uuid is very slow. You are right, Id may be a bad decision for the API and because we have other performance constraints, its not required.

So, my final statement: The API shoud use the caches.uuid as query parameter and give the developer the decision to use optionally caches.wp_oc where required. If have done good experience in my company with the id-class-parameter as explained in the last posting.- if you use it: great. If not ... no problem for me.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 24. August 2011, 07:20:08
Hm, I am not sure that what we want to exchange between OC Nodes use API. I think that it will better to keep individual changes between OC Nodes like different type of cache etc. One of main task is at current is How to users form others Nodes can write logs on other OC Sites without create new account. If I want find cache in Germany I can see on my cache maps on OC PL cahces located in Germany and when click on icon on map I can see properites cache on OC DE. When I found this cache I want to write log to this cache I must do this on OC DE but I want use my userrname sp2ong from OC PL and I don't want create account on OC DE fro make log. OC DE can verify my account on OC PL (maybe by API OAuth) OK I enter log  on OC DE and one thning what users need to have information about found, not found ??? in my OC PL profile for own statistics. Maybe API can get this info from OC DE to OC PL sql table cache_ocnodes (it is table to store base information about caches founde ?? by local users on other OC Nodes to create information in user profile. It is simple and we can not exchange many information between OC NOdes and this allow to have OC US own cache types, similar OC PL and OC DE. If OC PL user want create cache located in Germany must do this on OC DE becasue users must agree with local rules OC DE Node. For this reason OC PL can not create cache located outside  own country becasue we are decide that OC PL is local Geocaching services for polish users localed in Poland but at current OC PL users can register cache on OC PL located any country but it maybe better we can cahnge this like on OC US where during register cache user can select only Canada, US, Mexico.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 24. August 2011, 07:27:43
In our cache_status we have (...)

Okay, thank you for explanation.
The differences are in id 6 and 7.
We dont use 4 in oc.de (however, it may be the same as id 7).

For future flexibility, the OKAPI code should not rely on the id's.

SET NAMES 'utf8';
DROP TABLE IF EXISTS `cache_status`;
CREATE TABLE `cache_status` (
  `id` tinyint(3) unsigned NOT NULL auto_increment,
  `name` varchar(80) NOT NULL,
  `trans_id` int(10) unsigned NOT NULL,
  `de` varchar(60) NOT NULL COMMENT 'obsolete',
  `en` varchar(60) NOT NULL COMMENT 'obsolete',
  `allow_user_view` tinyint(1) NOT NULL,
  `allow_owner_edit_status` tinyint(1) NOT NULL,
  `allow_user_log` tinyint(1) NOT NULL,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `name` (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='static content' ;

INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('1', 'Ready for search', '531', 'Kann gesucht werden', 'Ready for search', '1', '1', '1');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('2', 'Temporary not available', '532', 'Momentan nicht verfügbar', 'Temporary not available', '1', '1', '1');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('3', 'Archived', '496', 'Archiviert', 'Archived', '1', '1', '1');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('4', 'Hidden by approvers to check', '533', 'Von den Approvern entfernt, um geürpft zu werden', 'Hidden by approvers to check', '0', '1', '0');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('5', 'Not yet available', '534', 'Noch nicht veröffentlicht', 'Not yet available', '0', '1', '0');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('6', 'Locked, visible', '821', '', '', '1', '1', '0');
INSERT INTO `cache_status` (`id`, `name`, `trans_id`, `de`, `en`, `allow_user_view`, `allow_owner_edit_status`, `allow_user_log`) VALUES ('7', 'Locked, invisible', '822', '', '', '0', '0', '0');

Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 24. August 2011, 07:35:54
Hm, I am not sure that what we want to exchange between OC Nodes use API.

I am sure we want to exchange at least:

user accounts basic info like name
user account authentication info, if user previously allowed it
table caches, caches_logs, pictures, cache_desc, coordinates, cache_attrib

We should begin with user, when it works, extend to replicate caches/cache_logs/cache_desc/coordinates/pictures an exchange cache_logs in the last step.

Maybe it is required that small servers only exchange user and basic cache-info for performance reason and database size.

But if we dont get a real replication running, we can never compete with gc.com.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 24. August 2011, 20:51:36
OK I have instaled latest version OKAPI on OC US and we can test on 2 sites this OKAPI

http://opencaching.pl/okapi/introduction.html
http://opencaching.us/okapi/introduction.html
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: RVRoadTrip am 24. August 2011, 21:04:06
On OCUS,we have cache types that other nodes do not have. Will these be ignored by the API?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 24. August 2011, 21:21:53
There will be only 4 "offical" cache types. All other types will display too, but they won't be documented.

BTW, the installation on opencacing.us is missing some files. It won't work properly.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 24. August 2011, 21:31:27
Now is upload full OKAPI package.

But short lin without www don't working and for test we are must use www.opencaching.us/okapi instead opencaching.use/okapi

I don't know where is problem
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 24. August 2011, 21:32:50
Does it matter?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 24. August 2011, 21:35:54
Tested OAuth and some other stuff at opencaching.us/okapi - everything seems to be working correctly.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 24. August 2011, 22:48:58
Published a new version. Now users and log entries are referenced by their UUIDs, instead of IDs. Internal IDs are hidden from an external developer.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 25. August 2011, 10:10:58
On page http://www.opencaching.pl/okapi/signup.html in form to get API Key exist star for  "Your application name*" "Contact email address*" it maybe better add information to that filed with * are required. and maybe suggest in filed "Your application name*" you can use username on OC Node
At current this page is available for all and I am not sure that it is OK because we want to have API for OC users and maybe better it to access to this page must be after login to OC Node to get API KEy for this reason we can get user_id for API Key and temporary we can put username "Your application name*" which user can change  to filed and simple add page to manage API Key by user to display existing API Key or regenerate new key. At current if I I lost information about my APi Key how to recovery this key ?

What do you think ?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 25. August 2011, 11:21:31
If you loose an Consumer Key you should generate a new one. Also, you should be able to generate many Consumer Keys - each of your applications may have a separate one.

It could be done the way you suggest, but I don't see a reason big enough to actually make me want to change it.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 25. August 2011, 12:15:52
What do you use for i18n? There's a builtin PHP function for that http://www.php.net/manual/en/function.gettext.php but you don't seem to use it. I found this instead: lib/languages/*. Can you explain me how does your i18n work? (in Poland AND in Germany).
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 00:07:56
Published a new version. Now users and log entries are referenced by their UUIDs, instead of IDs. Internal IDs are hidden from an external developer.

Great :)
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 26. August 2011, 03:13:48
Great :)

I thought you'll like it ;)

BUT - uuids of geocaches WILL be hidden. Geocaches will be referenced ONLY by their code, which is also unique.

Oliver, can you tell me about your locale/internationalization (i18n) support?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 26. August 2011, 07:00:18
OK OKAPI on OC US is updated to latest version

Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 10:41:02
I thought you'll like it

Maybe we can use UUID für cache-types and attributes, too. Especially with attributes, there are differences ... attribute id 7 may not be the same on oc.pl and oc.de. With an uuid, we could assign the "official uuid" of the attribute for each node to the attributes that match the "official attribute rules".

Zitat
Oliver, can you tell me about your locale/internationalization (i18n) support?

Summary:
1. Define texts to translate in .tpl or .php file (surround them by {t}...{/t} or in php with $translate->t(...)  )
2. Call the administration page and click scan source codes
3. The "scan" will push all transations into mysql table with reference where it is used
4. Translate all sentences with HTML-gui or use XML-export/import. As i remember, Olof added export-files in same format as oc.pl has.
5. Clear the cached-files after XML import
6. Cache files will be newly generated from mysql tables
7. smarty-template-system uses $translate->t() in background
8. $translate->t() uses php standard gettext()
9. You can also modify the SQL query to translate directly within the database request, which is very fast. E.g. table attribute has "name" column and "trans_id". With trans_id you can join to translation table and if no translation is found, use "name" column (use LEFT-JOIN and SQL-function-IFNULL() for that purpose)

Let me know if you need further infos or if you have problems setting up a testing environment. If you want to use gettext() directly, look into translate.class.php ... i am stripping out linespaces and linefeeds before storage of the string.

I did some screenshots in Sept. 2010:

(http://85.10.195.222/oc/oc-trans-1.PNG)
(http://85.10.195.222/oc/oc-trans-2.PNG)
(http://85.10.195.222/oc/oc-trans-3.PNG)
(http://85.10.195.222/oc/oc-trans-4.PNG)
(http://85.10.195.222/oc/oc-trans-5.PNG)
(http://85.10.195.222/oc/oc-trans-6.PNG)
(http://85.10.195.222/oc/oc-trans-7.PNG)
(http://85.10.195.222/oc/oc-trans-8.PNG)
(http://85.10.195.222/oc/oc-trans-9.PNG)
(http://85.10.195.222/oc/oc-trans-10.PNG)
(http://85.10.195.222/oc/oc-trans-11.PNG)
(http://85.10.195.222/oc/oc-trans-12.PNG)
(http://85.10.195.222/oc/oc-trans-13.PNG)
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 26. August 2011, 12:22:52
> Maybe we can use UUID für cache-types and attributes, too.
> Especially with attributes, there are differences ... attribute id 7
> may not be the same on oc.pl and oc.de. With an uuid, we could
> assign the "official uuid" of the attribute for each node to the
> attributes that match the "official attribute rules".

I hope there will be no need to do this. I decided to simply use
"strings" for dealing with such cases.

For example, there are 4 "official" cache types: "Traditional",
"Multi", "Quiz" and "Virtual". Developers are warned, that there are
also plenty of undocumented ones. OKAPI will simple referer to them by
their name. So, currently there is no need for IDs/UUIDs here (and I
hope there won't be one).

> 6. Cache files will be newly generated from mysql tables

What exactly are those "cache files"? Are there gettext's .po/.mo
files?

> 8. $translate->t() uses php standard gettext()

That sounds promising. But I'm not sure - from where exactly does
gettext get the translations from?

> Let me know if you need further infos or if you have problems
> setting up a testing environment. If you want to use gettext()
> directly, look into translate.class.php ... i am stripping out
> linespaces and linefeeds before storage of the string.

I can't use your $translate global (because there is no such class in
other OC installations), but I am planning on using php's gettext. And
I am wondering if you'd want to connect it (via
http://www.php.net/manual/en/function.bindtextdomain.php) with your
own translations directory.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 12:45:14
> 6. Cache files will be newly generated from mysql tables

What exactly are those "cache files"?

- "Smarty compiled templates" (see smarty manual)
- "Smarty cached templates"
- menu structure, generated from database table sys_menu,
- .po/.mo files for each language, generated from database table sys_trans and sys_trans_text

Zitat
Are there gettext's .po/.mo files?

yes, as you see in the screenshots.

Zitat
> 8. $translate->t() uses php standard gettext()

That sounds promising. But I'm not sure - from where exactly does
gettext get the translations from?

From the .po/.mo files. And these files are (re)generated from the database after upgrade of the sources.

Zitat
> Let me know if you need further infos or if you have problems
> setting up a testing environment. If you want to use gettext()
> directly, look into translate.class.php ... i am stripping out
> linespaces and linefeeds before storage of the string.

I can't use your $translate global (because there is no such class in
other OC installations), but I am planning on using php's gettext. And
I am wondering if you'd want to connect it (via
http://www.php.net/manual/en/function.bindtextdomain.php) with your
own translations directory.

oc.nl and oc.se has it, too. You can view the source from GIT repostory.
See http://forum.geocaching-network.com/index.php?topic=1359.0

Yes, we use bindtextdomain()

Our prepare_gettext()-function is used to ignore changes in html-template files, that dont need a new translation.

function load_gettext()
{
global $cookie, $opt, $rootpath, $locale;

$locale = $cookie->get('locale');
if (!isset($opt['locale'][$locale]))
$locale = $opt['template']['default']['locale'];
$opt['template']['locale'] = $locale;

bindtextdomain('messages', $rootpath . '/cache2/translate');

// setup the PHP locale
setlocale(LC_MONETARY, $opt['locale'][$locale]['locales']);
setlocale(LC_TIME, $opt['locale'][$locale]['locales']);
if (defined('LC_MESSAGES'))
setlocale(LC_MESSAGES, $opt['locale'][$locale]['locales']);

// no localisation!
setlocale(LC_COLLATE, $opt['locale']['EN']['locales']);
setlocale(LC_CTYPE, $opt['locale']['EN']['locales']);
setlocale(LC_NUMERIC, $opt['locale']['EN']['locales']); // important for mysql-queries!

textdomain('messages');
}

/* strip whitespaces
*/
protected function prepare_text($text)
{
$text = mb_ereg_replace("\t", ' ', $text);
$text = mb_ereg_replace("\r", ' ', $text);
$text = mb_ereg_replace("\n", ' ', $text);
while (mb_strpos($text, '  ') !== false)
$text = mb_ereg_replace('  ', ' ', $text);

return $text;
}
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 26. August 2011, 12:52:39
>> That sounds promising. But I'm not sure - from where exactly does
>> gettext get the translations from?
>
> From the .po/.mo files. And these files are (re)generated from the
> database after upgrade of the sources.

:) Sounds okay.

But why haven't you used xgettext and poeditor?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 13:05:37
:) Sounds okay.

But why haven't you used xgettext and poeditor?

I am no Unix-geek. I wanted to have the translation strings in database to allow translation directly within SQL. Each developer knows how to write SQL queries. But only a few developer know about details of gettext(). The framework abstracts complexity and even non-developer can translate everything without learning gettext (by html-frontend or xml files). I am also able to translate single strings into other languages than the standard frontend language (e.g. generating stats-image indepent of current user language). It works like a charme, so no need to change it.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 26. August 2011, 13:12:14
> I am no Unix-geek.

Nor am I. But there are Windows ports too.

> I wanted to have the translation strings in database to allow
> translation directly within SQL. Each developer knows how to write
> SQL queries. But only a few developer know about details of
> gettext(). The framework abstracts complexity and even non-developer
> can translate everything without learning gettext (by html-frontend
> or xml files).

I think you missed the fact that there are Windows tools for editing
.po files: http://www.poedit.net/screenshots.php

Your solution is quite nice, though a bit uncommon.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 14:38:12
Hi wrygiel,

now i have read your documentation at http://opencaching.pl/okapi/

Very well documented, good work! Function list is good selected for the beginning and the specification of fields-list is a good choice to reduce unwanted traffic.

Next step is to test the okapi-code on the new GIT-code.

Below is the list of my thoughts, while i have read the documentation. None of it is realy critical and i think you have realy done a good job. The only thing we should talk about is the is_admin-flag for the user.

Another question: Does the API code require access to the real OC.de-source e.g. with the current upload-images-directory or is it possible to install the API on a second webserver? We have virtual server infrastructure with 2 physical hosts. Installing a new virtual host with PHP 5.3 requirement but not move the oc.de-production-server to it will be easier. If required, i could map the oc-htdocs with NFS to that host.

We could add a service-locator-info-file e.g. http://www.opencaching.de/okapi.xml. Inside this file, we can redirect to other host e.g. okapi.opencaching.de/... or http://www.opencaching.de/okapi/... That could also simplify future upgrades, host-reinstallations and load-balancing.

Once i have time for migration, i willl then move the production oc-source to the new installed virtual host ...


Kind regards,
Oliver

-----
services/caches/geocache

- founds/notfounds ... we have a table stat_caches, so you dont need to recalculate these values each time. I am not sure if this table was introduced before or after oc-pl-merge.
- rating ... as i know, oc.pl has different rating system. recommendations sounds like the one we have.
- images ... we have a is_hidden-flag. A user may add images that are only referenced in HTML for decoration. These images should not be shown in the image list, but may be downloaded to decorate the html properly.

services/caches/formatters

- gpx ... Do you use search.gpx.inc.php for formatting or do you have own code?

services/caches/search

- all ... because of timeouts, memory limits or mysql locking issues, i think returning realy all geocaches is not a good idea. I dont tested it, so it may work good, but i think having no limit given will result to failures in future, so the API signatures should handle this from beginning ... could you use the LIMIT-syntax of mysql with start-parameter? e.g. SELECT * FROM caches ORDER BY wp_oc LIMIT 1000, 100

- size ... "caches with no container" could be size=0

- radius in /nearest ... we have a limit of 150km because calculation gets unefficient - then the developer should switch to bbox.

services/logs

- logs should also include any limit and start-parameter. E.g. a smartphone-application that shows the latest logs from caches/geocache and has a "show more" button. Show-more will show the next 10 logs. If the smartphone receives all logs of the caches, traffic may be too much. Our cache with most finds at oc.de has 750 finds (i've seen some at gc.com with >3000). Average log text length at oc.de is 300 chars. With suername, uuid and other infos, average bytes per log will be 400 bytes. 400 byte * 700 are 280 kByte in worst case.

- submit ... why dont allow HTML logs?

serice/users/user

- is_admin ... this should only be querable for the own user account for security reason. Also note that we have split admin priviledges into different levels.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 16:39:46
Comment from germany end user forum:
http://www.geoclub.de/viewtopic.php?p=907982

bbox in format W,S,E,N would be compatible with OpenStreeMap-XAPI
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 26. August 2011, 20:37:43
Ok we have 3 OC Nodes running with OKAPI :-)
Now http://www.opencaching.org.uk/okapi is ready
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 22:38:32
If given cache code does not exist, the method will respond with a HTTP 400 error.

The API also responds with this code, when i dont specify cache_code in caches/geocache-call
Maybe you can choose another code?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 26. August 2011, 23:13:22
Is there any way to validate the comsumer_key?
The response of the service is the same when i have invalid consumer_key or invalid cache_code.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 27. August 2011, 10:08:54
> Another question: Does the API code require access to the real
> OC.de-source e.g. with the current upload-images-directory or is it
> possible to install the API on a second webserver? We have virtual
> server infrastructure with 2 physical hosts.

Currently, OKAPI is a plugin for OC code. It is not a standalone
application. It could be made standalone, but I am not sure if this is
a good way to go. Currently OKAPI may take advantage of the existing
OC code.

Are you quite sure making OKAPI a standalone app is a good idea?

On the other hand, relaying on existing code might be a mistake.

> services/caches/geocache
>
> - founds/notfounds ... we have a table stat_caches, so you dont
> need to recalculate these values each time. I am not sure if this
> table was introduced before or after oc-pl-merge.

I don't recalculate either, we have caches.founds and caches.notfounds
fields.

> - rating ... as i know, oc.pl has different rating system.
> recommendations sounds like the one we have.

This could be troublesome. I thought the rating system will be a
common between all OC installations. You will probably have to create
a patch for OKAPI with some "site-specific conditions".

> - images ... we have a is_hidden-flag. A user may add images that
> are only referenced in HTML for decoration. These images should not
> be shown in the image list, but may be downloaded to decorate the
> html properly.

Have you checked the OKAPI code? Here is the query I use:
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/geocaches.php?r=80#160
Aren't you talking about the "display" column?

> services/caches/formatters
>
> - gpx ... Do you use search.gpx.inc.php for formatting or do you
> have own code?

My own:

http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/formatters/gpx.php?r=80
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/formatters/gpxfile.tpl.php?r=80

> services/caches/search
>
> - all ... because of timeouts, memory limits or mysql locking
> issues, i think returning realy all geocaches is not a good idea. I
> dont tested it, so it may work good, but i think having no limit
> given will result to failures in future, so the API signatures
> should handle this from beginning ... could you use the LIMIT-syntax
> of mysql with start-parameter? e.g. SELECT * FROM caches ORDER BY
> wp_oc LIMIT 1000, 100

I do not return all of them. There is a limit of 500 (100 by default).
The docs were misleading.

> - size ... "caches with no container" could be size=0

Added a new issue:
http://code.google.com/p/opencaching-api/issues/detail?id=52

> - radius in /nearest ... we have a limit of 150km because
> calculation gets unefficient - then the developer should switch to
> bbox.

How inefficient? Why?

Actually, I don't like the limit. If it turned out to be inefficient,
I would try to think of some heuristics to cope with that.

> services/logs
>
> - logs should also include any limit and start-parameter. E.g. a
> smartphone-application that shows the latest logs from
> caches/geocache and has a "show more" button. Show-more will show
> the next 10 logs. If the smartphone receives all logs of the caches,
> traffic may be too much. Our cache with most finds at oc.de has 750
> finds (i've seen some at gc.com with >3000). Average log text length
> at oc.de is 300 chars. With suername, uuid and other infos, average
> bytes per log will be 400 bytes. 400 byte * 700 are 280 kByte in worst case.

You're right. But I will probably have to make this parameter
optional.
http://code.google.com/p/opencaching-api/issues/detail?id=53

> - submit ... why dont allow HTML logs?

http://code.google.com/p/opencaching-api/issues/detail?id=54

> serice/users/user
>
> - is_admin ... this should only be querable for the own user account
> for security reason.

Why? Identities of admins are hidden?

Currently I use this only in one place:
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/logs/submit.php?r=80#78

> Comment from germany end user forum:
> http://www.geoclub.de/viewtopic.php?p=907982
>
> bbox in format W,S,E,N would be compatible with OpenStreeMap-XAPI

I can't change it now. 2 days ago OKAPI was officially released. Now
we have to be backward-compatible.

> The API also responds with this code, when i dont specify cache_code
> in caches/geocache-call Maybe you can choose another code?

No. Have you ever used some APIs? The 400 code is a standard "bad
request" response. Both of these requests are "bad requests".

> Is there any way to validate the comsumer_key? The response of the
> service is the same when i have invalid consumer_key or invalid
> cache_code.

Which service? Does this service require authentication?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 27. August 2011, 10:21:06
Currently, OKAPI is a plugin for OC code. It is not a standalone
application. It could be made standalone, but I am not sure if this is
a good way to go. Currently OKAPI may take advantage of the existing
OC code.

Are you quite sure making OKAPI a standalone app is a good idea?

No. Later, when you add function for data modification (beside "new log") e.g. change cache status or description, you will need the object from the busniess layer. With a standalone application, you will have duplicate validation logic.

What i mean is: Create a clean copy of the OC-source on a second host (api-host) and install OKAPI there. On that copy, you will have no file system access to user uploaded images, user stat-pictures etc. But the user app can load that images with a http-request to the www-host. So i dont think thats a real problem.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 27. August 2011, 10:29:02
> No. Later, when you add function for data modification (beside "new
> log") e.g. change cache status or description, you will need the
> object from the busniess layer. With a standalone application, you
> will have duplicate validation logic.

I think so too.

> What i mean is: Create a clean copy of the OC-source on a second
> host (api-host) and install OKAPI there. On that copy, you will have
> no file system access to user uploaded images, user stat-pictures
> etc. But the user app can load that images with a http-request to
> the www-host. So i dont think thats a real problem.

OKAPI does not store any cached data in the file system. But, on there
other hand, your "business layer" - might.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 27. August 2011, 10:30:41
This could be troublesome. I thought the rating system will be a
common between all OC installations. You will probably have to create
a patch for OKAPI with some "site-specific conditions".

I would prefer a "feature configuration file" in official repository code.

$opt['have_rating'] = true|false;
$opt['have_cache_found_stat'] = true|false;

Zitat
> - images ... we have a is_hidden-flag. A user may add images that
> are only referenced in HTML for decoration. These images should not
> be shown in the image list, but may be downloaded to decorate the
> html properly.

Have you checked the OKAPI code? Here is the query I use:
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/geocaches.php?r=80#160
Aren't you talking about the "display" column?

No, i did not check the source code until now.

Zitat
> services/caches/formatters
>
> - gpx ... Do you use search.gpx.inc.php for formatting or do you
> have own code?

My own:

http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/formatters/gpx.php?r=80
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/caches/formatters/gpxfile.tpl.php?r=80

okay.

Zitat
> services/caches/search
>
> - all ... because of timeouts, memory limits or mysql locking
> issues, i think returning realy all geocaches is not a good idea. I
> dont tested it, so it may work good, but i think having no limit
> given will result to failures in future, so the API signatures
> should handle this from beginning ... could you use the LIMIT-syntax
> of mysql with start-parameter? e.g. SELECT * FROM caches ORDER BY
> wp_oc LIMIT 1000, 100

I do not return all of them. There is a limit of 500 (100 by default).
The docs were misleading.

What do you think about the "start-at"-paramter?

(i will continue when i am back at home)
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 27. August 2011, 10:33:17
> What i mean is: Create a clean copy of the OC-source on a second
> host (api-host) and install OKAPI there. On that copy, you will have
> no file system access to user uploaded images, user stat-pictures
> etc. But the user app can load that images with a http-request to
> the www-host. So i dont think thats a real problem.

OKAPI does not store any cached data in the file system. But, on there
other hand, your "business layer" - might.

Ah, once you will add "image upload" feature (maybe iphone-app "log with picture), we will need a solution for that. But your answer is good, then i can install it on a new host with PHP 5.3 support when we fixed the compatibility issues.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 27. August 2011, 11:10:38
> I would prefer a "feature configuration file" in official repository code.
>
> $opt['have_rating'] = true|false;
> $opt['have_cache_found_stat'] = true|false;

Okay. I will implement this on Monday or Tuesday.

> What do you think about the "start-at"-paramter?

It depends on context.

I wouldn't encourage developers to download our entire database. OKAPI
is meant primarilly as a tool for online access. If developers want to
have dumps of data for offline use, I think we should think of a
special method for that - it would return a static JSON/XML/SQL dump,
generated once a day.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 27. August 2011, 20:50:12
Okay, back at home :)

I tried to implement VB.NET client library yesterday night and stucked when coming to oauth-login ...

> - radius in /nearest ... we have a limit of 150km because
> calculation gets unefficient - then the developer should switch to
> bbox.

How inefficient? Why?

I am not sure. I tested some things in 2005 and result was "limit it to 150km and you have no problem". For larger search radius, a bbox-search is most often appropriate, too. You know, for real radius search, there is a lot of sin() and cos()-calculation and processing time grows in square.

Most efficient way for distance calculation i found for mysql:

SELECT %distance AS distance {...} FROM caches WHERE lat>%min_lat AND lat<%max_lat AND lon>%max_lon AND lon<%max_lon HAVING distance<%max_distance ORDER BY distance ASC

where:
%distance is the formula returned by getSqlDistanceFormula()
%min_lat is the value returned by getMinLat()
%max_lat is the value returned by getMaxLat()
%min_lon is the value returned by getMinLon()
%max_lon is the value returned by getMaxLon()

With that formular, mysql performs a bbox-search first. That is fast. And only for that results, it calculates the distance formula. That is accurate.

Would be nice to that query pattern. In germany we already had some database performance issues in burst times before we offloaded search queries to our mysql slave server (on the second physical host).

Zitat
> serice/users/user
>
> - is_admin ... this should only be querable for the own user account
> for security reason.

Why? Identities of admins are hidden?

In general, we do not hide the real accounts behind our admins as groundspeak does. But i think its not required to make the is_admin queryable for everyone and in an bruteforce-attack, the first goal is to find a priviledged account. I have no problem if the API reports that the logged in user has admin priviledges.

Zitat
Currently I use this only in one place:
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/services/logs/submit.php?r=80#78

Ah, okay ... in the new code, there is a cache-business-layer-class which has property allowLog(). It would be a dream to have one code for all nodes ;)

Zitat
> Comment from germany end user forum:
> http://www.geoclub.de/viewtopic.php?p=907982
>
> bbox in format W,S,E,N would be compatible with OpenStreeMap-XAPI

I can't change it now. 2 days ago OKAPI was officially released. Now
we have to be backward-compatible.

If you think the suggestion of the geoclub-user is good, you could add a second supported input format:
- existing one with | seperated
- new one with , seperated

This should be backward-compatible.

Zitat
> The API also responds with this code, when i dont specify cache_code
> in caches/geocache-call Maybe you can choose another code?

No. Have you ever used some APIs? The 400 code is a standard "bad
request" response. Both of these requests are "bad requests".

I have used lots of APIs. And every good API gives a precise reason why the call failed. However, i did not use any JSON/REST-API until now. I worked with SOAP exceptions-model.

Zitat
> Is there any way to validate the comsumer_key? The response of the
> service is the same when i have invalid consumer_key or invalid
> cache_code.

Which service? Does this service require authentication?

It was services/caches/geocache

Thank you for accepting most of my points as an "issue". I know it is many "should and could and i would do in another way" - but again, you API looks good and i will try to get it running with our new code. But i need to be sure that i dont install any code at oc.de that does not fully comply with our busines logic.

I want to test some calls with my .NET-client-library (and test oauth) and then i will try to assist you with the "feature configuration file". solar22 is in vacation for a few weeks, as i remember.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 28. August 2011, 02:45:23
> I tried to implement VB.NET client library yesterday night and
> stucked when coming to oauth-login ...

Yeah, OAuth isn't easy.

> In general, we do not hide the real accounts behind our admins as
> groundspeak does. But i think its not required to make the is_admin
> queryable for everyone and in an bruteforce-attack, the first goal
> is to find a priviledged account. I have no problem if the API
> reports that the logged in user has admin priviledges.

Okay. You convinced me.
http://code.google.com/p/opencaching-api/issues/detail?id=57

>>> bbox in format W,S,E,N would be compatible with OpenStreeMap-XAPI
>
>> I can't change it now. 2 days ago OKAPI was officially released. Now
>> we have to be backward-compatible.
>
> If you think the suggestion of the geoclub-user is good, you could
> add a second supported input format:
> - existing one with | seperated
> - new one with , seperated
>
> This should be backward-compatible.

True, but it would be REALLY ugly.

>>> The API also responds with this code, when i dont specify
>>> cache_code in caches/geocache-call Maybe you can choose another
>>> code?
>
>> No. Have you ever used some APIs? The 400 code is a standard "bad
>> request" response. Both of these requests are "bad requests".[/quote]
>
> I have used lots of APIs. And every good API gives a precise reason
> why the call failed. However, i did not use any JSON/REST-API until
> now. I worked with SOAP exceptions-model.

But OKAPI *does* respond with a detailed error message! 400 is only a
HTTP response code, the message is the response *content*.

>>> Is there any way to validate the comsumer_key? The response of the
>>> service is the same when i have invalid consumer_key or invalid
>>> cache_code.
>
>> Which service? Does this service require authentication?
>
> It was services/caches/geocache

I don't believe you ;) What was this response exactly? These are
two completelly different errors and I'm quite sure they are reported
with similarly different error messages.

BTW, I am also not home. I will be more available after Monday.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 08:28:58
> I have used lots of APIs. And every good API gives a precise reason
> why the call failed. However, i did not use any JSON/REST-API until
> now. I worked with SOAP exceptions-model.

But OKAPI *does* respond with a detailed error message! 400 is only a
HTTP response code, the message is the response *content*.

Ah, okay! Then i have to look, how to get the resoponse-content in .NET Framework, after an Exception was thrown. I checked all properties of the .NET exception object, but did not find it. Thank you for pointing me to that.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: sp2ong am 28. August 2011, 11:20:19
I have see that application c:geo is opensource https://github.com/cgeo/c-geo-opensource and maybe it is easy add ? OKAPI support to this application
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 13:57:54
Ah, okay! Then i have to look, how to get the resoponse-content in .NET Framework, after an Exception was thrown. I checked all properties of the .NET exception object, but did not find it. Thank you for pointing me to that.

Works now :)
Thank you!
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 14:09:08
Catching Exceptions:
Do i have to parse the entire content of the response, or could we introduce a prefix?

Example:

For cache_code-invaild, i get currently the response "Parameter 'cache_code' has invalid value: This cache does not exist."

If you send me "CacheCodeInvalidException: Parameter 'cache_code' has invalid value: This cache does not exist.", i could check for string before : and throw exactly that exception. This would also enable us to throw localized exceptions.

Or should be use a JSON-structured-response?

My current code (entire string is compared):
Friend Class KnownExceptions
Public Const CacheCodeInvalid As String = "Parameter 'cache_code' has invalid value: This cache does not exist."
Public Const ConsumerKeyInvalid As String = "Parameter 'consumer_key' has invalid value: Consumer does not exist."

Public Shared Function GetKnwonException(ByVal ex As BadRequestException) As Exception
Select Case ex.Message
Case CacheCodeInvalid
Return New CacheCodeInvalidException(ex.Message, ex)
Case ConsumerKeyInvalid
Return New ConsumerKeyInvalidException(ex.Message, ex)
Case Else
Return ex
End Select
End Function
End Class
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 16:50:33
A real issue: I dont get okapi running on my development machine, because okapi requires that OC source code is installed in DOCUMENT_ROOT of apache. Now i need to reconfigure everything to support virtual hosts on my development machine :(
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 19:25:06
Okay, okapi is now running in my development machine ...
... i have modified the geocaches-routine to support our database layout (via Configuration class).

I send it with a private message to you.
Maybe you want to modify some naming conventions etc.

Please, let me know if that is a way to implement database differences directly within okapi.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 28. August 2011, 21:52:15
I submitted my first issue directly to google-code :)
http://code.google.com/p/opencaching-api/issues/detail?id=58
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 28. August 2011, 22:45:40
> Catching Exceptions:
> Do i have to parse the entire content of the response, or could we
> introduce a prefix?

What you want is for OKAPI to return a parsable (and extendable)
object with the details of the exception, instead of a single message.

In fact, I thought of that, but I didn't think anyone would be
interested in such a fine-grained exception control. Every bad request
is still a bad request, I am not sure if these additional exception
classes would actually do you any good. They all will be subclassing
BadRequest, and BadRequest is probably the only thing you will need to
catch (there is no need for special handling for other subclasses of
this exception).

I added an issue, but with a low-priority for now.
http://code.google.com/p/opencaching-api/issues/detail?id=59

> A real issue: I dont get okapi running on my development machine,
> because okapi requires that OC source code is installed in
> DOCUMENT_ROOT of apache. Now i need to reconfigure everything to
> support virtual hosts on my development machine :(

You are probably right. I haven't tested it in any other
configuration. Feel free to send me a patch.

> Okay, okapi is now running in my development machine ... ... i have
> modified the geocaches-routine to support our database layout (via
> Configuration class).
>
> I send it with a private message to you. Maybe you want to modify
> some naming conventions etc.

Okay, I'll look into it.

> I submitted my first issue directly to google-code :)
> http://code.google.com/p/opencaching-api/issues/detail?id=58

Thanks ;)
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 29. August 2011, 01:06:35
1.

I don't want for the configuration file to be kept within the "okapi"
directory. This could complicate updates (currently okapi can be
updated by just deleting it, and loading a new copy). Also, from what
I know, all OC settings are kept in the lib/settings.inc.php - OKAPI
will try to read it from there.

2.

You proposed two options: caches_stat_option and caches_have_votes.

Oc.pl uses caches_stat_option=1 and caches_have_votes=1.
Oc.de uses caches_stat_option=2 and caches_have_votes=0.

But I think there are no installations which will use any OTHER
combinations of these two flags. Am I correct? Do you think there will
EVER be such installations?

If not, I would still prefer to have it simpler:

if (oc.de) { sql(..) } else { sql(..) }.

I know having tons of settings seems professional, but I'd like to
keep things simple. SQL queries dependant on such multiple flags are
not simple at all.

As I understand, there are only two major OC ports - yours and ours.
Every other OC is based on oc.pl OR oc.de. That's why, the "if
(oc.de)" solution seems the right one here.

Having other settings would create an illusion, that other OC nodes
could use some of oc.pl code, and some of the oc.de code. But I think,
this *IS* an illusion. I think no one - except possibly us - will ever
try to merge any parts of oc.pl and oc.de.

I used the "if (oc.de)" approach (at least for now). This will
simplify any changes you will have to make to get it working on oc.de.

3.

Added some docs.


See here:
http://code.google.com/p/opencaching-api/source/browse/trunk/okapi/settings.php
and here:
http://code.google.com/p/opencaching-api/source/detail?r=81
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 29. August 2011, 08:26:35
Zitat von: wrygiel
Having other settings would create an illusion, that other OC nodes
could use some of oc.pl code, and some of the oc.de code. But I think,
this *IS* an illusion. I think no one - except possibly us - will ever
try to merge any parts of oc.pl and oc.de.

No one is OlofL  8)
http://forum.geocaching-network.com/index.php?topic=1203.0
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 29. August 2011, 11:39:38
No one is OlofL  8)
http://forum.geocaching-network.com/index.php?topic=1203.0

:) Can you tell me, in short, what's the project's intent? I haven't read that topic, but I assumed it was about merging your databases and moving oc.se to oc.de. Now, as I understand, the project is an attempt to include SOME (?) changes from oc.de into an oc.pl-based distribution?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 29. August 2011, 12:37:26
Zitat von: wrygiel
Can you tell me, in short, what's the project's intent?

short: Merging oc.pl and oc.de code.

long:
Zitat
The outline of this subproject is to:
- Create a branch (se-dev) from the oc.de source (oc-server). From a version control perspective this can be seen as a private branch for isolated development. The word "private" only means that the oc.de (oc-server) is unaffected during the development.
- Add features from the current oc.se source into the new se-dev-branch
- During this time work can still be done on the oc.de source (oc-server) and this can be merged to the new se-dev-branch.
- When the new se-dev-branch is stable enough for production oc.se will go live on this
- After the go live, this branch will be reintegrated back in the oc.de source (oc-server) and both oc.de and oc.se will go live on a common source.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 29. August 2011, 15:25:05
So, after the project finishes:
- oc.de will contain some additional features from oc.se/oc.pl,
- oc.se will start using oc.de code,
But:
- oc.pl (and some others) will still be using "old" oc.pl code (unless they choose to switch completely to the oc.de branch).

Am I correct?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 29. August 2011, 15:36:56
Zitat von: wrygiel
- oc.de will contain some additional features from oc.se/oc.pl

Yes, we allready got some oc.pl-features due to this merge, e.g. the personal cache note.


Zitat von: wrygiel
- oc.se will start using oc.de code

Better wording would be "using oc.de/oc.pl-code mix".


Zitat von: wrygiel
- oc.pl (and some others) will still be using "old" oc.pl code (unless they choose to switch completely to the oc.de branch).

Yes, I think so.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 29. August 2011, 16:03:27
>> - oc.se will start using oc.de code
>
> Better wording would be "using oc.de/oc.pl-code mix".

What I meant was: Oc.se will start using code from oc.de official
repository. (Oc.de will contain the oc.pl+oc.de merged code be then.)

>> - oc.pl (and some others) will still be using "old" oc.pl code
>> (unless they choose to switch completely to the oc.de
>> branch).
>
> Yes, I think so.

Then, in the end, we might say that we still end up with two branches:
"oc.de" (merged one) and "oc.pl". But with one big difference: "oc.de"
will have many features from "oc.pl", and the switch from "oc.pl" to
"oc.de" will be easier.

Hopefully, when you finish this merge, oc.pl (and others) will choose
to switch to oc.de branch. And ONLY THEN all OCs will have a common
repository.

Currently OKAPI uses conditions such as this one:

if (Settings::get('OC_BRANCH') == 'oc.de') {
   ...
} elseif (Settings:get('OC_BRANCH') == 'oc.pl') {
   ...
}

I think, we need a third value for your project. We might call it
'oc-merge-project':

if (Settings::get('OC_BRANCH') == 'oc-merge-project') {
   ...
} elseif (Settings:get('OC_BRANCH') == 'oc.de') {
   ...
} elseif (Settings:get('OC_BRANCH') == 'oc.pl') {
   ...
}

When oc.de decides to adopt your merge branch, it will switch their
OC_BRANCH value to 'oc-merge-project', and we will be able to remove
'oc.de' parts from OKAPI. Then (I hope), oc.pl (and others) will do
the same thing.
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 29. August 2011, 17:12:42
1.

I don't want for the configuration file to be kept within the "okapi"
directory. This could complicate updates (currently okapi can be
updated by just deleting it, and loading a new copy). Also, from what
I know, all OC settings are kept in the lib/settings.inc.php - OKAPI
will try to read it from there.

Okay.

2.

You proposed two options: caches_stat_option and caches_have_votes.

Oc.pl uses caches_stat_option=1 and caches_have_votes=1.
Oc.de uses caches_stat_option=2 and caches_have_votes=0.

But I think there are no installations which will use any OTHER
combinations of these two flags. Am I correct? Do you think there will
EVER be such installations?

Yes, it could. And i hope so. The migration of new oc.pl-code to git-repository is still in progress. The votes might get migrated in future. oc.pl might decide to migrate some of our features in future.

Of course, its not "oc.de code", we should name it "git-repository".

We have at least 3 repositories:

1) oc.pl-svn. They have own development team and its used by several sites.

2) The git-repository with different branches (based on oc.de-svn)
=> oc.se/oc.no runs from that repository
=> all future development will be done in git-repository

3) oc.de-svn with different branches
=> The oc.de production server will be moved to git repository, as soon as the code is compatible. We have most of that migration done. So at the end of this year, oc.de may use the source from the git repository
=> oc.nl runs also from that repository and after oc.de runs from git, i will try to migrate oc.nl to the git-repository

Zitat
As I understand, there are only two major OC ports - yours and ours.
Every other OC is based on oc.pl OR oc.de. That's why, the "if
(oc.de)" solution seems the right one here.

Both solutions have disadvantages ...
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: oliver am 29. August 2011, 17:19:13
So, after the project finishes:
- oc.de will contain some additional features from oc.se/oc.pl,
- oc.se will start using oc.de code,
But:
- oc.pl (and some others) will still be using "old" oc.pl code (unless they choose to switch completely to the oc.de branch).

Am I correct?

oc.se already runs the code from git-repository. The git-repository is base on smarty. So all php-files migrated are "totaly different". There are still some unported files like "search.php" and "newcache.php".

What oc.se did is, that they used the "new" oc.de code with smarty and integrated some new oc.pl features. Most important: the additional waypoints. The additional waypoints are online/runnning at oc.se and oc.de, but oc.nl does not have it until now, because of some local modifications that make update complex ...
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: mic@ am 06. Juli 2014, 21:57:42
There are some features which OKAPI are not supporting,
e.g. picture upload within log or maintenance logging (german "kann gesucht werden").
Is it planned to integrate these features soon?
Titel: Re: OKAPI Project - API for OpenCaching sites
Beitrag von: wrygiel am 06. Juli 2014, 22:37:27
You can submit all feature requests here: https://code.google.com/p/opencaching-api/issues/list - most feature requests get implemented after some time (some take days, others take months). Also note, that not all OC sites use the latest version of OKAPI.