Survey Coordinate entry. RTK GPS and even drone, help...

Help, problems, suggestions about Waterski Homologation software
emmanuel
Site Admin
Posts: 276
Joined: Fri May 20, 2005 8:34 am
Contact:

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by emmanuel »

If you find such a tool, I am VERY interested in it, at least to check what I program...
or to use, if I don't success in that programming !

Not a lot of thought on that type of survey, as I am not surveying any waterski courses anymore.
It is surely one of the possibility for the future.

Emmanuel
tjones99
Posts: 18
Joined: Thu Aug 09, 2012 7:37 pm

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by tjones99 »

I was at the site today and wanted to redo the coordinates I had that were Float (less accurate) vs Fix from the original survey and it was really calm and things were working well so I just redid the survey completely. Attaching the CSV file.

There were issues with elevation in the previous survey, these seem to be very close in this survey.

At first glance I think this may be a good result!
Attachments
Summerski_V2.zip
(1.07 KiB) Downloaded 3278 times
tjones99
Posts: 18
Joined: Thu Aug 09, 2012 7:37 pm

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by tjones99 »

Status update... if you have read this far then you are interested and I should follow through and keep things up to date...

Last Saturday I redid a survey at the lake. new data, better data (in RTK terms the data points were solution status FIX vs float meaning they should be cm accurate...)

Since then, lots of discussion with Emmanuel and lots of education on coordinate conversions from Lat Lon height to XYZ and UTM coordinates and short answer is I had no idea how to move from one to the other but I am piecing it together with help from many people like Emmanuel and a few people on the Emlid Reach forums (The GPS units are Emlid Reach units).

Epiphany was I needed to convert from LLH (Latitude Longitude height) that the GPS unit survey output produces to UTM coordinates (ie XY on a flat plat for a particular region on the globe) and I found NCAT by NOAA does that. Took the conversions and plugged them into the homologation software and got my first survey.

Here is a picture of the rover on the lake over a gate ball. The pink arrow shows where my Base unit is located on the top left corner of the steel shipping container.
Rover.jpg
Rover.jpg (261.78 KiB) Viewed 74292 times
tjones99
Posts: 18
Joined: Thu Aug 09, 2012 7:37 pm

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by tjones99 »

Just a further follow up post.

First, HUGE thanks to Emmanuel who did all the heavy lifting to integrate this new approach into the software! I was expecting this to be a long term project and it is already available.

Since it is now possible, a little more info is needed if anyone is interested in trying this approach.

First a warning... you can't just go out with your handheld GPS unit and survey your course. Regular GPS is nowhere near accurate enough. RTK or Real Time Kinematics reduce the GPS error by referencing a fixed location. Its still not as simple as it sounds... it takes two GPS units, one at a fixed location called the base and one that moves from ball to ball called the rover. The base unit communicates with the rover and they figure out where the rovers is by looking at where the GPS signals tell the base it is located and where the SAME GPS SATELLITES say the rover is... all this happens in real time using high end GPS receivers and a radio link.

The units I am using are Emlid Reach RTK units. https://emlid.com/
It looks like the original Reach units I am using have been replaced by a more "finished" unit called the Reach M+
You need two of these to survey, so that's $650US for the pair with antennas. You need radios that plug into them and those are about $50US a pair they are known as SIK radios or 3DR radios. So the emlid solution is a $700US solution. Not cheap, but not as expensive as a Total Station survey instrument with prism and tripod.

The big plus is this is a one person solution. Park the base unit on shore. Take the floating unit out and survey. All you do is place the rover over each ball, collect the position, move on to the next ball. The traditional Total Station survey is a minimum two person task and the instrument costs more. Another plus is that this can be relatively simple, you don't have to spend a lot of time trying to figure out how to use a Total Station. Additionally, I have found that the best time to view the iPad is NOT mid day in bright sunlight, so surveying late in the evening when its too dark to ski is ideal! The lake is usually calm and no skiers to make waves and mess up your survey!

The Emlid Reach units have a lot of "smarts" in them, they have their own wifi capabilities, all you need to do the survey is a wifi capable device with a web browser such as a laptop or a tablet or even a smartphone, though having a larger display like a tablet helps.

There is some DIY work required to get the whole setup wired. Radios have to be connected to each unit. The GPS antennas have to be mounted on metal "ground planes" and the rover is best if made pretty waterproof. Then there are the configuration settings on the Reach units themselves to turn one into the base and the other into the rover and to set up the radio communication for the GPS corrections to pass back and forth between the units.

Once set up, then you simply connect to the rover's wifi and using a web browser (Safari/Chrome/Edge etc) you connect ot the rover and us the built in survey tool.

In the survey, you collect a point at each ball. To use the drag and drop integration that Emmanuel added to the GPS input, you must name each point the same as the balls in the element you are surveying. ST1 St2 PG1 PG2 B1 B2 G1 G2 etc... The survey from the reach units is a CSV or comma separated value file and the header line identifies the name, latitude, longitude and height from the survey and these will import seamlessly into the Homologation software.

This is still just an overview... questions? Fire away.
jtim3032
Posts: 5
Joined: Sat Feb 23, 2019 12:51 pm
Location: Amersham, UK

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by jtim3032 »

Hi, I am new to this forum...

Before I knew of this software, I also started last October developing a homologation system based on RTK GPS. It is now working well (at least for slalom) and I will post some more information in due course. But rather than re-invent the wheel as it were for software to cover a whole event, it would make sense for my data collection systems to export a file that could be imported direct into Emmanuel's software.

I gather you already have "drag & drop" for individual fields but the import of a complete CSV file (one file for each course) would be a lot simpler. I could also UDP data to any port of our choosing which perhaps would be even neater?

I couldn't find any import function on the current software version so I would value your thoughts :)

Thanks in advance

James
emmanuel
Site Admin
Posts: 276
Joined: Fri May 20, 2005 8:34 am
Contact:

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by emmanuel »

Hi James,

Nice to know that some other people are also starting using RTK GPS device for waterski surveys.

You need to know that I do NOT have any RTK GPS device.
I did only develop this software, and Terry from Canada was doing the survey on his lake !

For now, there is no good import function. csv works somewhat.
For any automatic download from RTK GPS device (if this what you mean with UDP), I might be able to implement something, if I get a description of the interface (technical guide of the device, or something similar) !

As you mention you have already done a RTK GPS software for slalom, it would be nice to compare results of both your and mine software, with the same data input.
Just to proof each other ! You might be able to do that with some surveys that you might have done.
As fas as I know, the GPS part of the software was only tested by Terry.

Emmanuel
jtim3032
Posts: 5
Joined: Sat Feb 23, 2019 12:51 pm
Location: Amersham, UK

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by jtim3032 »

Hi Emmanuel

Yes I have already compared calculation results using yours and my software and gratifyingly they exactly match! I assume therefore you are also using Vincenty formulae for the maths :) I hope to do a side by side survey soon with a Total Station in order to compare results with old and new systems.

Just for info, all GPS receivers (RTK capable ones included) can stream ascii data out in NMEA format, using whatever comms capability they support. My receivers can stream over Bluetooth, TTL and USB so makes them quite flexible.

If you wanted to go down that road in the future, you would only need to parse 1 or 2 of the NMEA sentences to get Latitude & Longitude data from the serial feed. The feed is continuous however, so you would need to provide a "Mark" button or similar to enter the data into whichever field had focus.

In the meantime, to make things more convenient and practicable, I have nearly finished an Android app which would collect the data in the field and then export it to my software and hopefully into yours eventually using a straight forward comma delimited text stream. Alternatively it could be saved to file in CSV format and then loaded into your software from file. Maybe this is the easier method in the short term?

I am happy to provide the data in whatever format would be easiest for you to implement.

In a day or so, I should be able to send you a sample CSV file. The field naming will match your software exactly,
eg "G1,<G1Lat>,<G1Lon>,G2,<G2Lat>,<G2Lon>,G3,<G3Lat>,<G3Lon>,etc...etc"

Rather than store an Altitude for every field, since the lake is flat, I just store a single Altitude for the survey. Possibly the best place in the file for that might be in a header line at the beginning of the file?

James
emmanuel
Site Admin
Posts: 276
Joined: Fri May 20, 2005 8:34 am
Contact:

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by emmanuel »

Hi James,

I did not know about Vincenty formulae !
I had some other sources, which might rely on the very same formulas.
Important is that we get the exact same results !!
It is a relief that your results are the same as mine !

I also don't know anything about NMEA format, but if that is a standard format that GPS system can output, I could eventually implement that.
So, yes, if you can send me some data example, it would be useful.

For the short term, yes, CSV file, with same buoys name and same column name should do the job.
(if not, please send me the file).

In the my homologation software, I already implement some communication (Rs232c over Bluetooth or USB) between theodolite and software.

I could implement another communication for GPS, but will have no chance to test it, as I don't have such device (and will not, as I am not homologating anymore).

Emmaanuel
jtim3032
Posts: 5
Joined: Sat Feb 23, 2019 12:51 pm
Location: Amersham, UK

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by jtim3032 »

Hi Emmanuel

Yes it was a relief to me also when the numbers came out the same :lol:

I have attached a sample CSV file from a survey of our lake. Some of the buoys are well out of tolerance :)

James
Attachments
RWSC_Forward.zip
(506 Bytes) Downloaded 3217 times
emmanuel
Site Admin
Posts: 276
Joined: Fri May 20, 2005 8:34 am
Contact:

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by emmanuel »

Thanks.
I will have a look when I will be less busy.

Emmanuel
tjones99
Posts: 18
Joined: Thu Aug 09, 2012 7:37 pm

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by tjones99 »

Sorry for the long silence on this topic from my end... and also sorry for this long post... but I am back looking at this further... and wanted to share my findings as well as my ponderings...

James, I also read your post of Mon Feb 25, 2019 12:07 pm and some of this response may help...

Yes GPS devices frequently use NMEA data streams, and streams they are... 1-5 updates per second and often many NMEA "sentences" (as they are called) per update... so a "mark" approach is required... and just to reiterate, not just any GPS device is accurate enough for this task... thats where the RTK or Realtime Kinematics, comes in and provides the necessary position corrections to get centimeter accuracy instead of the usual meter accuracy (which is useless for this task) (which makes on wonder how Stargazer and Zero off get all the accuracy they claim but I suspect it is relative accuracy and not absolute and therefore suitable for the task... I digress...)

Thats part of the reason that I chose the Emlid devices as they are really a complete solution because they actually include built in survey software that can export the survey data in the format that Emmanuel has added to his drag and drop import into the program. (still can't believe how fast you did both the coordinate conversion and the import feature Emmanuel...)

Looking at the Emlid web site presently there have been some changes since my last post. the devices I am using have been replaced with a new version (they had to, an Intel component they were using went End of Life) call the Emlid reach M+ and they also now sell the additional "bits" I used to connect them so you can literally add all you need to your shopping cart at the Emlid web site to get you a survey setup. You would need two Emlid reach M+ receivers, two antennas, two LoRa radios. Attached is what the PDF of the cart would look like. $900US in total... not cheap but cheaper than any total station and easier to use IMHO. In addition to the above gear you need a mobile device and I prefer a tablet but a phone will work in a pinch.

At this stage I would not run out and buy gear as I am doing some further testing after researching another RTK approach that uses only a single unit and the corrections come from an internet source canned NTRIP. Still unclear if that will work in areas where NTRIP sources are not close to the site... more research to be done... but a quick test between two ground points on my driveway using NTRIP corrections gave me a repeatable 19.47m distance when expecting 19.5m (and frankly, those marks were from rope measurements and with tape measure stretch etc... could easily be out 3cm...).

So the task as soon as our ice is out (still thick enough to drive on the ice at this point so will be a month at least) is to head to the lake, do a traditional survey and a GPS survey using two RTK GPS units during ideal conditions on the same day just to double confirm that the results are identical, then do another using a single unit and NTRIP corrections. Busy day if all goes well.

But while that has been going on I have been looking at various alternatives to the Emlid RTK GPS gear to see if a more economical approach is possible. Ardusimple (https://www.ardusimple.com/simplertk2b/) now has simpleRTK2B product that uses a new multi band GPS module that should improve speed to get a fix and they have a full kit similar to the Emlid kit I mentioned that is 550 Euros. But theirs does not include the survey component that is built into the Emlid product... it would require programming which I am looking into. Similar to Ardusimple is a new board from SparkFun (https://www.sparkfun.com/products/15136) called SparkFun GPS-RTK2 Board - ZED-F9P (Qwiic). SparkFun does not have a complete kit package but their products are very modular so it should be easy enough to built a shopping list from their site. they sell everything necessary. Again, programming required.....

This is where the ponderings start...

Looking at the SparkFun modules, I started to think that a dedicated waterski product may evolve here to simplify the homologation task as well as the task of setting up a site in the first place... Up here setup is easy in winter... walk out, measure accurately on the ice, cut holes, drop anchors, come spring add balls... (OK, its fricking freezing work, hard getting blocks out on a snow covered lake, and to chainsaw 26+ holes through 10-20 inches of ice to drop anchors through is no picnic either...). So I was pondering using the RTK GPS device to set up the course first... build a custom device with a touch screen LCD. Go to a new lake and give the device two points as a reference. the two points are on the boat path at the two ends of the course (i.e. the course balls will be centered between those two initial points). Once you give it two points it calculates where all the balls are for the type of course you are setting up (slalom, jump, barefoot jump etc...) and one at a time the device "leads" you to where the ball needs to be. We use a barrel float device in summer to float an anchor until we get it into position and then lower it when in position. Once it is just touching the bottom you fine tune it (bounce it) to the exact position. So the custom GPS device would have a compass as well and you would set it on top of your "float" and move it until the unit says you are in position then lower the anchor. The LCD would let you select which ball you wanted to install and then just tell you which way and how far it needed to go to be in correct position. A lot easier than the survey approach that requires repeated measure/enter into the program/move/fudge iterations to get a ball in the right place.

Thinking the same device could also be used to check an existing course. Pick two balls in an existing course that you believe are in the correct position and it will tell you where the rest should be relative to those two. Should also be useful for things like checking total length of a cable course to see if you are stretched or short...

And it could survey a full course and download a CSV for the homologation program.

Thinking it is just a combination of two RTK GPS units, two LoRa radio links, a touch LCD (or maybe a tablet and a wifi link and a mobile app as that may be more flexible long term) and an Arduino to do the calculations etc...

So thats where I am at present.

TJ
Attachments
Cart - Emlid Store.pdf
(963.59 KiB) Downloaded 3171 times
jtim3032
Posts: 5
Joined: Sat Feb 23, 2019 12:51 pm
Location: Amersham, UK

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by jtim3032 »

Hi All

It's about time for an update from me and again apologies in advance for the long post :)

Like TJ, I have set up an RTK system for homologation. The two receivers I started with are Alpha Polaris RTK units, either one of which can act as a base station or rover. As TJ has described, the base unit streams error data to the rover in order that it can do the RTK maths to achieve 1-2cm precision relative to the base position.

I tried various methods of getting the error data to the rover and started by using a local radio setup (like TJ) but I found the radios which were rated up to 1km in range only achieved 300-400metres in the real world. This was mainly because the transmission baud rates had to be quite high in order to not lose any of the error data and this lowers the range.

I then moved up to much better radios (the excellent RFD900), up to 40km in range and these worked well until I tried to survey a site near a large airport and found the radio interference from nearby radars again dropped the useable range to about 500 metres.

Lastly I tried transmitting the base error data in RTCM3 format (via my laptop) to my online server which relayed the data (using STRSVR from the RTKLib library) to my mobile phone which in turn used a Bluetooth connection to pass the data on to the rover and receive position data back. In the end this has proved to be the most reliable method but involved quite a lot of work developing the Android app to do the necessary stuff. This work however has now provided unexpected dividends as I shall explain.

I found that the Polaris receivers would take up to half an hour (sometimes longer) to get to a full RTK Fix mode where all "integer ambiguities" were resolved and 1-2cm precision achieved. This was workable but certainly not ideal. In the end I decided to upgrade to new units (the simpleRTK2B) which are based on the new chip available from u-blox, namely the ZED-F9P which TJ has already referred to. This new chip is a dual frequency receiver and as such can process data from more than one satellite constellation, eg it can process GPS, GLONASS AND Beidou or Galileo as opposed to just GPS+GLONASS. Also receiving data on two frequencies from the same satellite allows some RTK maths that makes resolving the integer ambiguities much simpler and faster.

In short, the difference in performance is astounding!. TJ, do not delay, just get some! They take a bit more configuration, especially the base but it is well worth it. I now get to RTK Fix mode in under 2 minutes from a cold start and retaining RTK Fix is very robust.

The Android app had been designed to collect all the survey data and then export it as a CSV file for use in Emmanuel's software (or other as desired). However, with the new receivers, I soon realised that the reliable retention of RTK Fix even when the boat is travelling at 58kph has made it possible to extend the app to be able to monitor the boat position within the boat lane when driving for slalom. To achieve this though I found that the integrity of the GPS antenna signal was crucial. In the boat, the antenna had to be mounted about 10cm above the top of the windscreen to get a good clear view of the sky. The app now shows the boat position and indicates with beeps etc whenever the boat deviates more than the required 20cm from the centre line.

Below are a couple of screenshots from the app.
Image

Image

In case anyone is interested also, I decided to invest in a 3D printer in order to build a support for placing over a buoy when homologating its position. Whilst this arrangement is very functional and self centres over buoys of different sizes within the allowable range (there is an inner cone shape which allows for the different buoy diameters), I have since realised that for a working homologator travelling by plane, it is far too bulky to fit conveniently in a suitcase! I will be doing some redesigning in due course :)
Image

Cheers

James
tjones99
Posts: 18
Joined: Thu Aug 09, 2012 7:37 pm

Re: Survey Coordinate entry. RTK GPS and even drone, help...

Post by tjones99 »

It has been a while since there was any discussion on this but I recently re-visited this helping someone else and wanted to post an update.

While revisiting this I found that my original data no longer worked an working with Emmanuel found that when I was creating my Slalom course element in the Homologation program I was not selecting Decimal Degrees as the Unit of measure and this was causing an issue with my drag and drop. My data that had worked previously was showing large errors now. Emmanual has changed the program so that it is no longer required to select Decimal Degrees when using GPS Coordinates. Thanks Emmanuel.

I also ran into an issue with drag and drop not working at all immediately after a fresh install. This turned out to be because I had to run the installer as admin (I log in on a limited user account) and at the end when it launched the program that program is running as admin and you cannot drag and drop from one user login to another in Windows. Emmanuel has correct this so the installer automatically requests admin credentials and then at the end runs the program as the normal user so that is not an issue.

Re-reading this thread I realized there was some unanswered discussion about drag and drop. Worth making it perfectly clear. You can drag and drop a WHOLE FILE of CSV data and it imports correctly into the program as long as you have the column headers correct and the element names correct.

So for example here is some GPS data that when saved in a CSV file (Comma Seperated Values, file extension .CSV) can just be dragged and dropped into the data area of the homologation program and populates everything perfectly. If you look at the data below you will see that the data has longitude then latitude and you will notice that the lines of data are not in the order that the program expects but it understands based on column headings and element names and puts everything in the right place! So it doesn't matter what order you havecaptured your GPS data for your slalom course, it works.

I believe Emmanuel has also made the application work for other elements in all disciplines (Barefoot Jump for example) but I have not tested this yet.

name,longitude,latitude,height
G1,-80.13079501,43.48689849,285.2680591
G2,-80.13081452,43.48691591,285.2564233
S6,-80.1286911,43.48832017,285.2390565
B1,-80.13053906,43.48705487,285.2423197
B2,-80.13055679,43.48707039,285.2523259
B3,-80.13014973,43.48728988,285.2576841
B4,-80.13016738,43.48730632,285.2349231
B5,-80.12975846,43.48752621,285.1941413
G3,-80.1283334,43.48838786,285.2579258
G4,-80.12835302,43.48840501,285.2407853
S1,-80.13045722,43.48698315,285.249577
S2,-80.13024921,43.48737794,285.2550074
S3,-80.12967826,43.48745424,285.2382671
S4,-80.12947009,43.48784905,285.2369487
S5,-80.12889922,43.48792574,285.2537258
B6,-80.12977847,43.48754186,285.2624067
B7,-80.12937124,43.48776202,285.2628795
B8,-80.12938802,43.48777761,285.2446048
B9,-80.12898052,43.48799713,285.267863
B10,-80.12899927,43.4880135,285.2576286
B11,-80.12859153,43.48823301,285.2698787
B12,-80.1286091,43.48824911,285.2352768
Pg1,-80.13131739,43.48658229,285.246566
Pg2,-80.13133865,43.48659827,285.2480586
Pg3,-80.12781005,43.48870453,285.2658391
Pg4,-80.12783057,43.48872108,285.2286
Post Reply