OKAPI Project - API for OpenCaching sites

[wrygiel]

Tested OAuth and some other stuff at opencaching.us/okapi - everything seems to be working correctly.

[wrygiel]

Published a new version. Now users and log entries are referenced by their UUIDs, instead of IDs. Internal IDs are hidden from an external developer.

[sp2ong]

On page http://www.opencaching.pl/okapi/signup.html in form to get API Key exist star for  "Your application name*" "Contact email address*" it maybe better add information to that filed with * are required. and maybe suggest in filed "Your application name*" you can use username on OC Node
At current this page is available for all and I am not sure that it is OK because we want to have API for OC users and maybe better it to access to this page must be after login to OC Node to get API KEy for this reason we can get user_id for API Key and temporary we can put username "Your application name*" which user can change  to filed and simple add page to manage API Key by user to display existing API Key or regenerate new key. At current if I I lost information about my APi Key how to recovery this key ?

What do you think ?

[wrygiel]

If you loose an Consumer Key you should generate a new one. Also, you should be able to generate many Consumer Keys - each of your applications may have a separate one.

It could be done the way you suggest, but I don't see a reason big enough to actually make me want to change it.

[wrygiel]

What do you use for i18n? There's a builtin PHP function for that http://www.php.net/manual/en/function.gettext.php but you don't seem to use it. I found this instead: lib/languages/*. Can you explain me how does your i18n work? (in Poland AND in Germany).

[oliver]

Published a new version. Now users and log entries are referenced by their UUIDs, instead of IDs. Internal IDs are hidden from an external developer.

Great

[wrygiel]

Great

I thought you'll like it

BUT - uuids of geocaches WILL be hidden. Geocaches will be referenced ONLY by their code, which is also unique.

Oliver, can you tell me about your locale/internationalization (i18n) support?

[sp2ong]

OK OKAPI on OC US is updated to latest version

[oliver]

I thought you'll like it

Maybe we can use UUID für cache-types and attributes, too. Especially with attributes, there are differences ... attribute id 7 may not be the same on oc.pl and oc.de. With an uuid, we could assign the "official uuid" of the attribute for each node to the attributes that match the "official attribute rules".

Oliver, can you tell me about your locale/internationalization (i18n) support?

Summary:
1. Define texts to translate in .tpl or .php file (surround them by {t}...{/t} or in php with $translate->t(...)  )
2. Call the administration page and click scan source codes
3. The "scan" will push all transations into mysql table with reference where it is used
4. Translate all sentences with HTML-gui or use XML-export/import. As i remember, Olof added export-files in same format as oc.pl has.
5. Clear the cached-files after XML import
6. Cache files will be newly generated from mysql tables
7. smarty-template-system uses $translate->t() in background
8. $translate->t() uses php standard gettext()
9. You can also modify the SQL query to translate directly within the database request, which is very fast. E.g. table attribute has "name" column and "trans_id". With trans_id you can join to translation table and if no translation is found, use "name" column (use LEFT-JOIN and SQL-function-IFNULL() for that purpose)

Let me know if you need further infos or if you have problems setting up a testing environment. If you want to use gettext() directly, look into translate.class.php ... i am stripping out linespaces and linefeeds before storage of the string.

I did some screenshots in Sept. 2010:

[wrygiel]

> Maybe we can use UUID für cache-types and attributes, too.
> Especially with attributes, there are differences ... attribute id 7
> may not be the same on oc.pl and oc.de. With an uuid, we could
> assign the "official uuid" of the attribute for each node to the
> attributes that match the "official attribute rules".

I hope there will be no need to do this. I decided to simply use
"strings" for dealing with such cases.

For example, there are 4 "official" cache types: "Traditional",
"Multi", "Quiz" and "Virtual". Developers are warned, that there are
also plenty of undocumented ones. OKAPI will simple referer to them by
their name. So, currently there is no need for IDs/UUIDs here (and I
hope there won't be one).

> 6. Cache files will be newly generated from mysql tables

What exactly are those "cache files"? Are there gettext's .po/.mo
files?

> 8. $translate->t() uses php standard gettext()

That sounds promising. But I'm not sure - from where exactly does
gettext get the translations from?

> Let me know if you need further infos or if you have problems
> setting up a testing environment. If you want to use gettext()
> directly, look into translate.class.php ... i am stripping out
> linespaces and linefeeds before storage of the string.

I can't use your $translate global (because there is no such class in
other OC installations), but I am planning on using php's gettext. And
I am wondering if you'd want to connect it (via
http://www.php.net/manual/en/function.bindtextdomain.php) with your
own translations directory.

[oliver]

> 6. Cache files will be newly generated from mysql tables

What exactly are those "cache files"?

- "Smarty compiled templates" (see smarty manual)
- "Smarty cached templates"
- menu structure, generated from database table sys_menu,
- .po/.mo files for each language, generated from database table sys_trans and sys_trans_text

Are there gettext's .po/.mo files?

yes, as you see in the screenshots.

> 8. $translate->t() uses php standard gettext()

That sounds promising. But I'm not sure - from where exactly does
gettext get the translations from?

From the .po/.mo files. And these files are (re)generated from the database after upgrade of the sources.

> Let me know if you need further infos or if you have problems
> setting up a testing environment. If you want to use gettext()
> directly, look into translate.class.php ... i am stripping out
> linespaces and linefeeds before storage of the string.

I can't use your $translate global (because there is no such class in
other OC installations), but I am planning on using php's gettext. And
I am wondering if you'd want to connect it (via
http://www.php.net/manual/en/function.bindtextdomain.php) with your
own translations directory.

oc.nl and oc.se has it, too. You can view the source from GIT repostory.
See http://forum.geocaching-network.com/index.php?topic=1359.0

Yes, we use bindtextdomain()

Our prepare_gettext()-function is used to ignore changes in html-template files, that dont need a new translation.

Code: [Auswählen]

function load_gettext()
{
global $cookie, $opt, $rootpath, $locale;

$locale = $cookie->get('locale');
if (!isset($opt['locale'][$locale]))
$locale = $opt['template']['default']['locale'];
$opt['template']['locale'] = $locale;

bindtextdomain('messages', $rootpath . '/cache2/translate');

// setup the PHP locale
setlocale(LC_MONETARY, $opt['locale'][$locale]['locales']);
setlocale(LC_TIME, $opt['locale'][$locale]['locales']);
if (defined('LC_MESSAGES'))
setlocale(LC_MESSAGES, $opt['locale'][$locale]['locales']);

// no localisation!
setlocale(LC_COLLATE, $opt['locale']['EN']['locales']);
setlocale(LC_CTYPE, $opt['locale']['EN']['locales']);
setlocale(LC_NUMERIC, $opt['locale']['EN']['locales']); // important for mysql-queries!

textdomain('messages');
}

/* strip whitespaces
*/
protected function prepare_text($text)
{
$text = mb_ereg_replace("\t", ' ', $text);
$text = mb_ereg_replace("\r", ' ', $text);
$text = mb_ereg_replace("\n", ' ', $text);
while (mb_strpos($text, '  ') !== false)
$text = mb_ereg_replace('  ', ' ', $text);

return $text;
}


[wrygiel]

>> That sounds promising. But I'm not sure - from where exactly does
>> gettext get the translations from?
>
> From the .po/.mo files. And these files are (re)generated from the
> database after upgrade of the sources.

Sounds okay.

But why haven't you used xgettext and poeditor?

[oliver]

Sounds okay.

But why haven't you used xgettext and poeditor?

I am no Unix-geek. I wanted to have the translation strings in database to allow translation directly within SQL. Each developer knows how to write SQL queries. But only a few developer know about details of gettext(). The framework abstracts complexity and even non-developer can translate everything without learning gettext (by html-frontend or xml files). I am also able to translate single strings into other languages than the standard frontend language (e.g. generating stats-image indepent of current user language). It works like a charme, so no need to change it.

[wrygiel]

> I am no Unix-geek.

Nor am I. But there are Windows ports too.

> I wanted to have the translation strings in database to allow
> translation directly within SQL. Each developer knows how to write
> SQL queries. But only a few developer know about details of
> gettext(). The framework abstracts complexity and even non-developer
> can translate everything without learning gettext (by html-frontend
> or xml files).

I think you missed the fact that there are Windows tools for editing
.po files: http://www.poedit.net/screenshots.php

Your solution is quite nice, though a bit uncommon.

[oliver]

Hi wrygiel,

now i have read your documentation at http://opencaching.pl/okapi/

Very well documented, good work! Function list is good selected for the beginning and the specification of fields-list is a good choice to reduce unwanted traffic.

Next step is to test the okapi-code on the new GIT-code.

Below is the list of my thoughts, while i have read the documentation. None of it is realy critical and i think you have realy done a good job. The only thing we should talk about is the is_admin-flag for the user.

Another question: Does the API code require access to the real OC.de-source e.g. with the current upload-images-directory or is it possible to install the API on a second webserver? We have virtual server infrastructure with 2 physical hosts. Installing a new virtual host with PHP 5.3 requirement but not move the oc.de-production-server to it will be easier. If required, i could map the oc-htdocs with NFS to that host.

We could add a service-locator-info-file e.g. http://www.opencaching.de/okapi.xml. Inside this file, we can redirect to other host e.g. okapi.opencaching.de/... or http://www.opencaching.de/okapi/... That could also simplify future upgrades, host-reinstallations and load-balancing.

Once i have time for migration, i willl then move the production oc-source to the new installed virtual host ...


Kind regards,
Oliver

-----
services/caches/geocache

- founds/notfounds ... we have a table stat_caches, so you dont need to recalculate these values each time. I am not sure if this table was introduced before or after oc-pl-merge.
- rating ... as i know, oc.pl has different rating system. recommendations sounds like the one we have.
- images ... we have a is_hidden-flag. A user may add images that are only referenced in HTML for decoration. These images should not be shown in the image list, but may be downloaded to decorate the html properly.

services/caches/formatters

- gpx ... Do you use search.gpx.inc.php for formatting or do you have own code?

services/caches/search

- all ... because of timeouts, memory limits or mysql locking issues, i think returning realy all geocaches is not a good idea. I dont tested it, so it may work good, but i think having no limit given will result to failures in future, so the API signatures should handle this from beginning ... could you use the LIMIT-syntax of mysql with start-parameter? e.g. SELECT * FROM caches ORDER BY wp_oc LIMIT 1000, 100

- size ... "caches with no container" could be size=0

- radius in /nearest ... we have a limit of 150km because calculation gets unefficient - then the developer should switch to bbox.

services/logs

- logs should also include any limit and start-parameter. E.g. a smartphone-application that shows the latest logs from caches/geocache and has a "show more" button. Show-more will show the next 10 logs. If the smartphone receives all logs of the caches, traffic may be too much. Our cache with most finds at oc.de has 750 finds (i've seen some at gc.com with >3000). Average log text length at oc.de is 300 chars. With suername, uuid and other infos, average bytes per log will be 400 bytes. 400 byte * 700 are 280 kByte in worst case.

- submit ... why dont allow HTML logs?

serice/users/user

- is_admin ... this should only be querable for the own user account for security reason. Also note that we have split admin priviledges into different levels.