- -
UserInfo

Welcome Anonymous




Membership:
Latest: Floody65
New Today: 0
New Yesterday: 1
Overall: 888

People Online:
Members: 0
Visitors: 2
Bots: 13
Total: 15
Who Is Where:
Visitors:
01. News
02. Forums
 Bots:
01. Forums
02. Login
03. Forums
04. Forums
05. Forums
06. Forums
07. Login
08. Login
09. CPGNuCalendar
10. Forums
11. Home
12. Forums
13. Forums

Staff Online:

No staff members are online!
We have received
33113086
page views since
July 27, 2010

Hits New Today: 402
Hits New Yesterday: 21748

Server Time
25 November 2017 00:27:56 AEDT (GMT +11)
Q&A > GPS Tracker > Damn, we hit our free google API request limit last night > Community Forums > Mustang Forum Australia - Mustang Tech

View posts since last visit • View posts last 24 hours • View unanswered posts
Forum Index > GPS Tracker

Damn, we hit our free google API request limit last night Reply to topic

Go to page 1, 2 Next
Author Message
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


The google free API is limited to 2500 requests per day.
I use it to retrieve the full address of the GPS coordinates provided by the tracker.
Last night with George and Mike driving around and me working on my car, we hit that limit.

I've changed over to a paid account now, but that's not sustainable when providing a free service.
I'm going to have to test some of the free reverse geocoding providers and try to incorporate them into the system.

This only affects the address shown in the information bubble on the website and in the app.

What I might have to do in the end as more of us jump on is have the free tier use the free geocoders, and a "premium" service use the google service.
The problem with the free ones that I have looked at in the past, is they generally don't provide a full address down to the house number - just street/suburb.

Anyway... will see what I can find.


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

1 👍 / 0 👎


I will be interested in what you find as i have used openmaps lot so not look to much further.


I will someday think of something clever to say.

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


A lot of the free ones seem to use openstreetmap backend.
Were you getting street numbers out of it?


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


I not normally had to deal with street addresses - always tracking vehicles using lat/long data or items / asset or even locations of events all via direct lat/long data.
I have use Mapstraction API is an easy way to switch between the various APIs though you could go straight to OpenLayers so the API that has geocode()

geoadmin.github.io/ol3...a.Map.html

Then there is Pelias which might be a better option to provide the encoding

github.com/pelias/pelias

and use Openstreetmap for the mapping.

Any way I am interested in what you choose and what you learn.


I will someday think of something clever to say.

Last edited by boofhead on Sat Jan 14, 2017 2:11 pm; edited 1 time in total

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


I use mapstraction in the website itself to display the location of the lat/lon on the map.
It's actually the reverse geocoding within the tracking server (that actually talks to the trackers) that has hit the limit, not so much the mapping itself.

So the tracker sends it's lat/lon, I send that to a reverse geo provider, and they send back an address.

I've just developed code for locationiq.org/ and switched everyone over to that.
They do support house numbers, but that information hasn't been inserted into the openstreetmap database for Australia yet.
If you query a location in the UK for example, the house numbers are included.

So any paying customers can use the Google API, but freebies will use locationiq. Hopefully one day the Australian data for house numbers will be imported to openstreetmap and there will be no difference between the two providers from our point of view.


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


Great - was not aware of that service so great stuff.


I will someday think of something clever to say.

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


Just had a crazy thought.
I do so many lookups when people are driving around - why not store every lookup in my own database and query that database first.

Maybe that way I can switch back to using google for everyone, but will have to query google a lot less. Most people would go to pretty similar places often.
I wonder what the chances of hitting the same place would be though. Even if you're 5 meters away from last time, the coordinates would be different.

Maybe I can just say if it's within 10 meters, use it. Otherwise, do a new lookup. I think it's worth looking at anyway.


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


Caching lookups is an important technique to reduce load and improve performance. I am not so familiar with the software you have built that I could suggest anything specific though.


I will someday think of something clever to say.

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


Sun, 15 Jan 2017 00:38:03 +1100 1.129.97.44:64213 Found xxxxxxx, xxxxxxx NSW 2153, Australia in cache Database. Bypassing lookup.

Will be interesting to see how it goes.
I would still prefer to check if the location falls within a 20m radius of an existing location in the cache database, but I don't think that's too easy without querying every record each time.


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


So every GPS location the hardware collects and sends to the hybrid server the first thing it does it translate it to Street address using google services - this is what I think your saying?

You can do a local 20m compare using lat long data alone to existing cached entries and if one or more found then take the first and do no lookup. You could also decide to keep a last address looked up for each device and just use it for the next 10 mins and just append it on the current incoming GPS location. The address will be "Close to" but far fewer lookups - with both combined you will reduce street lookups considerably. I am not sure what you use the lookup street data for. I assume just display purposes. I would also age the cache entries say 30 mins so fresh ones appear from time to time. The gui could have the option to select an entry to manually do an address lookup if they want an accurate address for a location. This would go into the cache. You will need to limit the cache size so cache access performance cost stay low.


I will someday think of something clever to say.

View user's profile
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


Just to clarify I was thinking of a separate cache in memory not looking up the path records in the database - my suggestions will result in the path records becoming less accurate street address terms. It would be costly to scan all records to do the street address look up via an area match in GPS coordinates. Needs to be local and fast and use a far smaller data set.


I will someday think of something clever to say.

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


I will just see how it goes using my current method for now.
The problem with a short-term local cache is that unless the car is sitting at a set of lights, it will have moved a fair way by then (which is why you say it will be less accurate).
The openmaps database is under 60MB, so if I just store every looked up location, the DB shouldn't get too big. Over time, the external lookups should get less and less.

It will also save a bunch of lookups while our cars are sitting at home not moving for days.

I've switched everyone back to using google maps, so everyones trackers will be contributing to me building my own location data down to the house number level.


View user's profilePhoto Gallery
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


746 db hits in 7 hours. Not bad really - that number should continue to get bigger and bigger.

Still 889 external lookups in the same period.


View user's profilePhoto Gallery
boofhead
Mustang King

Offline
Joined: Nov 02, 2010
Posts: 4456
Location: Brisbane

0 👍 / 0 👎


So a 50% reduction. Not to bad if the performance in providing the data from your local database is reasonable. This is depends on people traveling the same path - which is true mostly. How many driving around for the 1600 data points?


I will someday think of something clever to say.

View user's profile
hybrid
Mustang King

Offline
Joined: Jul 27, 2010
Posts: 8372

0 👍 / 0 👎


Mike and gbx plus a few hits from my own house.


View user's profilePhoto Gallery
All times are Australia/Sydney
Go to page 1, 2 Next


Jump to:

TopPostersToday
   hybrid 
 Posts: 
 3 

   66STNG 
 Posts: 
 2 

   Husky65 
 Posts: 
 2 

   Nuts 
 Posts: 
 2 

   krisr 
 Posts: 
 2 

   lukep6470 
 Posts: 
 1 

LatestForumPosts
Last 10 Forum Messages

hybrids 1966 Coupe Project
Last post by nassi in Mustang Projects on Nov 24, 2017 22:40:35

Unable to post
Last post by hybrid in Website Help on Nov 24, 2017 18:45:55

Project Pheonix
Last post by Nuts in Mustang Projects on Nov 24, 2017 17:30:02

MSD 6AL-2
Last post by Husky65 in Mustang Parts for Sale on Nov 21, 2017 17:52:04

Husky's 65 Restomod
Last post by gbx78 in Mustang Projects on Nov 21, 2017 16:32:04

Astro's GG-0351 coupe build.
Last post by Astro in Mustang Projects on Nov 21, 2017 15:00:10

Scarlet
Last post by gbx78 in Mustang Projects on Nov 20, 2017 21:55:06

Project Lime A code.
Last post by Shaunp in Mustang Projects on Nov 20, 2017 21:46:48

56 effie.. FnL
Last post by nassi in Other Projects on Nov 20, 2017 20:51:04

Jacko's Project
Last post by Pinto-Pete in Other Projects on Nov 20, 2017 20:39:56

SocialMedia

Like this page? Google +1 it!