Sauf mention contraire dans les contenus, l'ensemble de ce site relève de la législation française et internationale sur le droit d'auteur et la propriété intellectuelle.
James Fee GIS Blog
Indoor mapping is the white whale of our Spatial IT industry. We’re always reading about how our smartphones will lead us to the best deals or how I can find the specific nail I need in Home Depot without having to ask anyone or walk down every aisle. They key to all this is essentially iBeacon.
You can search Google News for all the latest excitement on the concept but essentially it is a way for your phone to know where things are and for the vendors to know where you phone is through Bluetooth. Imagine walking into a store and getting alerts about your favorite beer being on sale and then the ability to navigate directly there. Sexy right? Plus we’ve been anticipating this happening for years. Except…
Google was set to launch a new product that added context to one of its most successful apps, Google Maps. But earlier this year, it was shut down by Alphabet CEO Larry Page, according to people familiar with the project.
Google Here worked by sending a notification to a smartphone user’s lock screen within five seconds of their entering a partner’s location. If the user clicked on the notification, a full screen HTLM5 “app” experience would launch. Google Here would know when to send the notification via Google Maps and beacons placed in the stores of participating partners. Google planned to supply the beacons to partners for the launch, according to the document. The experience could also be found by going to the Google Maps app.
Exactly what we though everyone wanted. In testing the application was deemed too invasive and Google feared no retailers would sign up. That’s right, Google didn’t think could get their partners to install cheap beacons in their stores AND they feared they were too big brother. Seems weird doesn’t it, if there is one company that can get companies to spend money on ads, it is Google. And since when did Google ever think pushing ads on us was “invasive”?
The magic about Google Here1 was that you didn’t need an app running for it to work. Think about that for a minute, ads would appear on your phone based on where you where and you didn’t need to opt in to get them. Now we see why Google was very concerned that Here was going to get a large backlash. Being able to push ads on users would have been something they really could have sold well to companies, I’m not sure there would be any fear of companies not wanting to push ads on us.
Beacons are still very important to Google. Their Eddystone project talks about lots of uses of beacons but not for ad delivery. Clearly there was feedback on this project and it jolted Google out of their normal sell more ads business model. I think beacons will be very valuable as they start appearing in more areas, but I for one don’t need to get an ad for fabric softener every time I walk into a Target.
Here as in not Here that was owned by Nokia↩
I am working on a project that needs to display all the neighborhood polygons in Baltimore City at one time. The file is relatively detailed… which mean that tons of unecessary polygon nodes are being sent from the backend, when, at the zoom level and the level of detail the map users need, the high level of detail is a total waste.
While there are some great hosted options to serve up complex GeoJSON, most of the time it is better served1 to simplify your data. Unless you’re surveying or involved with some sort of lawyer, even a bit of generalization is a good idea with online mapping. Chris does a great job showing how you can modify the tolerance to get your results to look great and save lots of bandwidth. If you’re a generalization newbie, you should read his example and get a better understanding of how it works.
And if you’re an Esri user, the same concepts can be used in their stack as well.
no pun intended↩
Yesterday I had a long post about GIS and version control. I mentioned Git in the article saying how maybe in the future Git would work with GIS files. A couple of people mentioned an article written by Gretchen Peterson titled, “Huge increase in shareability by combining Git and QGIS“.This isn’t the version control you’re looking for…
Gretchen does a great job showing how you can manage GIS projects with Git and I encourage everyone to read it. But keep in mind it isn’t version control how I was talking. Git doesn’t understand shapefiles and other binary GIS files. It will show that a shapefile was updated, but it won’t show what you updated in it, nor will it help reconcile updates. Gretchen is using Git to help her share projects with others which it does a great job. But it isn’t geodata version control and she outlines that clearly in the post.
We all would love to see Github support shapefiles, but I seriously doubt it will ever happen. For how you can use Github and GeoJSON files which isn’t half bad.
Today’s HWJF planning staff meeting was full of new ideas. The biggest one was that HWJF becomes a podcast. One of the biggest feedback requests has been to offer an audio only version of the hangouts for those who want to listen on their smartphones offline. I’ve explored this many times and never really got a good plan in place. But given that HWJF isn’t really visual in nature (looking at my grill for an hour has to be taxing), we’re going to convert HWJF into a podcast for season 4 arriving in October. If there is a visual need to have video, we’ll have special hangouts on the YouTube. What this means is it won’t be live anymore so you can’t point out how wrong I am until after the faux pas has passed.
What it will mean is the podcast should be more consumable and usable by everyone. This is an experiment so we’ll see how it goes moving forward. It will also allow more flexible scheduling of the podcast so we can have guests on who can’t attend during work hours.
Update: It appears that Boundless abandoned GeoGig to the Eclipse Foundation. Currently it shows no work has been done on GeoGig in the past 12 months. Time to assume our only hope is Github itself.
The crazy thing about GIS is that we never really take into consider version control with our data. Well we have a workflow, it usually entails putting a one at the end of a file name or calling it “temp” or “trash” while it works through analysis. Whenever I used to take over a project from someone, I’d take a quick look at the project folder and you’d see hundreds of seemingly orphaned GIS datasets just littering up the folder structure. And of course no documentation as to why there are there, how they were created and what the derivative products were created off of them.
When I joined WeoGeo many moons ago, that was one thing Paul Bissett always harped on and was something WeoGeo was trying to solve. WeoGeo was approaching it from a data sales marketplace end where data providers wanted to know what derivative works were being crated off their data. But for users, the same process was needed. We tried to pitch it to users but generally they didn’t see the value in keeping track of their datasets. I think this was shortsighted and I still believe that WeoGeo should have been the choice of every GIS professional to maintain an authoritative data library. But alas, GIS professionals didn’t care.
We’ve watched GeoGig by Boundless (It appears that they passed it on to the Eclipse Foundation last year. I hadn’t heard this, nor does the website show the update). for years, waiting to see if it will succeed. It provides the kind of revision control we’re used to with programming. Boundless has a pretty good graphic below that shows what’s going on from a GIS perspective.
Bingo right? Well wrong… While GeoGig is actually very impressive and takes into consideration all those weird things us GIS folks do, it won’t ever go anywhere. Without it being integrated into QGIS and ArcGIS Desktop users won’t be able to integrate it into their workflows. It is the same problem we ran into with WeoGeo Library, it’s just too hard to integrate it into ArcGIS Desktop without Esri doing it themselves.
But the QGIS tie is interesting. Boundless is behind GeoGig. They are also a big supporter of QGIS. GeoGig seems more tied in with Geoserver right now but editing is the big reason for GeoGig and let’s be honest. Most editing happens on the desktop. Boundless has been showing that QGIS and GeoGig work together but as I said above. Unless it is natively integrated into QGIS, it won’t have more than a niche uptake. At this point though, GeoGig is really our only big hope.
Esri has versioning on their geodatabase but it’s a nightmare. I’ve never had good luck with it but I’m willing to chalk it up to me not knowing a thing about what I’m doing. Geo-data is complex, Git works because it is so simple. Tracking simple changes in text for files. They have a hard enough time working with GeoRSS but you can see that it does work. I have to often wonder if GitHub might have GIS version control before GIS people get it working. Honestly that is what we want right? Github for GIS?
Well what could go wrong?
The B4UFLY app, aimed primarily at model aircraft enthusiasts, is designed to give users information about restrictions or requirements in effect at their current or planned flight location. The FAA expects the beta test will yield valuable data on how well B4UFLY functions, as well as uncovering any software bugs.
So first, it’s a private beta so you probably aren’t invited. You can email firstname.lastname@example.org if you want to give it a shot and ask to be in the private beta. Second, the app looks simple but I suspect the rules are going to be complex and thus the app will be less than easy to use. Still if you want to fly your drone, this could be the start of something good. Tomorrows, my birthday, feel free to buy me one.B4UFLY App on iPhone
If there is one thing that gets the GeoMarketers excited, it is natural disasters. Every year at this time we see hurricane trackers, 3D storm viewers and other “exciting” products to help protect us from the wrath of mother nature. This year Google is promoting their Personalized Storm Tracker.
The safety recommendations you receive will be tailored to reflect the current status of the event and your context. For example, if you search for a specific storm when it’s still several days away, you may see a map of the developing weather event and a recommendation to start preparing an emergency kit. If the storm is only hours away from your location, you might receive a reminder to start charging your phone in case power goes out. And if you search when the storm is nearby, you’ll get the most urgent information, like how to avoid injury from fast-moving water or flying debris.
I feel like these things are more trouble than they are worth. Last Sunday I was at Sky Harbor Airport when a dust storm alert went off on all the iPhones in the baggage claim area. The whole place sounded loud as the alerts went off warning us that wind and dust were headed our way. The result of this great warning, people making jokes. Sure an alert was issued, sure it probably is a safety thing that we all get these alerts on our phones. But the delivery isn’t personalized, it’s a broadcast message and then 10 minutes of people joking about the alerts.
At least in the USA, storms don’t sneak up on anyone. These products are great press but of little value.
I never “celebrated” 10 years of Spatially Adjusted mostly because I forgot about it. I was cleaning up the site earlier this week and noticed there was some good content back then, it definitely had a different tone but hey, I’m 10 years older now. I’m going to post a “best of” link every week to a 10-year-old article for the rest of the year. Some of it will be thought-provoking1 and some of it will be laughable. At any rate 10 years ago this week there were a couple posts about hurricane tracking that were interesting given that it was about Katrina, but this one caught my eye.
All the openness in the world won’t make any product successful, but listening to your customers will. The feeling that I’ve gotten from ESRI over the past year is that they have finally begun to realize that their road to continued success is supporting users like us. Don’t confuse the hype surrounding Google Maps/Earth with them being open and listening to their customers. There is no company that likes to hide behind their logo more than Google and they will do whatever it takes to not have to be open. There is a reason people are beginning to realize that Google is the next Microsoft (while Microsoft seems to have becomethe next IBM). Believe me, ESRI has a LONG WAY TO GO before they are as open as we’d all like them to be, but they do listen to their customers and that is a start.
Well the whole post is sort of like that, me claiming that Esri has been more open than Google or others. The context with this is they started allowing their employees to blog and contact people directly, it was a big shift from the traditional call a phone number support. So we were all so excited to see Esri employees blogging and responding to our articles. Well eventually it all collapsed into a corporate marketing blog cycle but at that moment it looks like we felt like Esri was changing.
disclaimer: probably not↩
Years ago in the Arc/Info world, we used to perform most of our geoprocessing in ArcInfo Workstation on Windows. But when we needed to really get work done, we’d use a HP-UX beast of a server to handle some of the more complex geoprocessing. It was really easy to do right, Esri even use to have some tools to help you accomplish this. I remember thinking that very soon we’d be able to offload most geoprocessing on remote constellations and then just get back the results. My personal workstation wouldn’t be bogged down with processing and the server would be doing what we paid good money for.
Well we didn’t know what we were talking about at the time was “GIS as a Service”. Mostly because we didn’t think of clouds anything more than rain makers. But the idea of offloading our geoprocessing was something to a person we’d wager would be built into GIS by now. Of course products like ArcGIS Server and FME Server can run processing remotely but it is not built into workflows. You have to go out of your way to author scripts that can handle this. I’m curious why things worked out this way.
It could be that with Arc/INFO on Unix going away there wasn’t servers that could handle geoprocessing. Or it could be that workstations these days are so fast that you don’t need to remote process. Maybe I’m just old and stuck in my ways that I want to use an Unix server for processing, maybe put a couple of Perl scripts in there and call it a day. But I think I’m disappointed that we just haven’t seen that much uptake on remote geoprocessing. The only workflow I’ve used this on that was supported by the software is authoring on FME Desktop and running those workbench scripts on FME Server.
I guess we always assume there will be flying cars and houses on the moon but we’re left with airport departure TVs that show the blue screen of death, smartphones that can be hacked with SMS and our credit cards being stolen left and right. The reality of GIS in 2015 is it is still enterprise work being done in a workgroup fashion. GIS isn’t taken seriously by IT because we don’t take ourselves seriously. Hiding in a corner “doing GIS” is how we’re seen by others. Time to break the mold.
Something I started in 20061 is still widely used. I created it originally as I was trying to create ArcGIS 9.1 Personal Geodatabases with ArcGIS 9.2. It wasn’t possible then to create older Geodatabases but Esri eventually added in functionality to create older versions. The reason we need these is that you can use older Geodatases in newer versions of ArcGIS but not the other way around. So if you are on ArcGIS 10.2 and your client is on ArcGIS 9.3, you’ll have a problem sharing data. But if you have a 9.3 version Geodatabase, then you can save your data to that version and share away.
I like this archive because each one of these Geodatabases was created with that version of the software. They will work perfectly since they are natively created. So next time you need to have a 8.3 Geodatabase2, you’ll have a native Geodatabase to work with. Bookmark and use!
Special thanks to @GIS_katie for providing the updated blank ArcGIS 10.3 File and Personal Geodatabases.
Well good news for those who want to help a down on its luck company like Google update their maps.
Google Map Maker, the tool which allows anyone around the world to contribute information to Google’s worldwide map, has re-opened in 45 countries after going live again in 6 countries two weeks ago. The product was temporarily shut down in May after it was discovered that some nefarious edits to the map, like geographic polygons shaped to depict an Android peeing on what is ostensibly an Apple logo, were being approved.
If you want to help Google, just go to Google Map Maker and start editing. Just know your edits will get locked up and used to make a ton of money. Here in the USA you can’t create polygons yet but I suppose that will be back soon.
Look I love iOS but I still use Google Maps as much as possible because it works better than any other mapping service out there. But I’m beginning to wonder what Google is thinking by adding some new features.
Now Google is looking to capitalize on this ongoing trend with a new feature in Google Maps that encourages users to share their “foodie pics” with others by posting the photo to Google Maps itself.
It could be that I live in a car town and navigation is the reason I use Google Maps but the idea that I would use my mapping app to take pictures of food is a bit out there. I mean don’t they have their own social media network to handle this? Oh right…
It’s always interesting to hear about the latest mobile mapping apps but I thought this was interesting.
Its signature TripTik is still going strong, the auto club says, even in the age of in-dash GPS and Google Maps. The TripTiks come free with AAA membership.
AAA is not only still making TripTiks but they are still giving them out free. Given the article doesn’t give ages but they sound from the older generation. TripTik is still around but it’s day is numbered.
Legacy GIS SystemWe were talking this weekend about how much serving up GIS data has changed in the past 3 years. GIS Server used to be so important to many of my friends companies to the point they spent tens of thousands of dollars on it a year. But no longer, each one said that they stopped paying for server because they all use other options. Now before I go on, I want to say this isn’t about sales data of Esri products. It’s more about changes in how people are sharing spatial data. Feel free to replace ArcGIS Server with your favorite GIS server package1.
I gave a talk years ago about something we did at the GNOCDC mapping recovery from Hurricane Katrina. You can see the slide deck here and watch the video here. Basically it was the seeds of what we are going through right now. It wasn’t that what we were doing back there was very unique, it was just a realization that GIS can’t be hosting “enterprise” data in a “workgroup” environment. Just like Katrina basically broke the GNOCDC GIS servers, it has become clear that there is almost no way for an organization to use classic GIS servers without putting a lot of load balancing and networking decisions in front of them.
For most companies this is just way too much infrastructure and licensing costs. We’ve seen the rise of CartoDB, Mapbox and ArcGIS Online2. Each has pluses and minuses and while there is overlap, they all do things unique to themselves. But what the big attraction for each is that you don’t have to manage the constellation yourself.
The biggest drawback each said was the unknown in licensing. Most hosted GIS plans are costed in ways that GIS people aren’t familiar with. Mapviews? Nobody has analytics on that until you put it in these services. 100,000 map views sounds huge doesn’t it? But how do you really know? Service credits? We’ve wondered what that even means for years. But I’d wager beers that even with the unknown, you’ll still save money over your ArcGIS Server license or other maintenance you pay for hosting your own GIS server.
We’re at a crossroads here. People have begun to start realizing standing up ArcGIS Server, Geoserver or other map servers makes little to no sense in the new marketplace. Paying for hosting maps is cheaper in the long run, has more availability and is easier to use that classic self hosted mapping solutions. ArcGIS Online for all it’s confusion is beginning to be leveraged by users and everyone I knew at the Esri UC knows what CartoDB and Mapbox do. Back in the old days of WeoGeo, we had to prove what we know now every day. The cost of “doing it yourself” is magnitudes higher than paying for hosting.
Tide is changing…
Since I’ve decided to break Hangouts with James Fee into Spring and Fall “seasons”1 the summer has been left to swimming and vacations. But with Fall around the corner 2 it’s time to get serious about scheduling the next batch of hangouts. I asked for feedback from people last season and it was a great help. For the fall though I’d love to interview people who haven’t been on the show before or are not as well-known. If you can email me with suggestions (heck include yourself if you want) I’d really appreciate it. My favorite Hangout from last spring was with Lyzi Diamond and I’d love to have more like that.
— Brian Timoney (@briantimoney) August 19, 2015
I hadn’t really thought of the article in that context, I was just looking at a quick way to turn a CSV into a GeoJSON file quickly. But let’s look at Brian’s point, is desktop GIS heavy?
I’ve maintained since Esri abandoned ArcInfo Workstation in the early 2000s, GIS has become difficult to use. Not in the sense that any idiot1 can click the next button, but the simple fact they have no idea what they’re doing. To accomplish this, Esri spent tons of R&D to make GIS as simple as drag a couple of layers to a dialog and just click next until you have an output. You don’t even need to understand the setting, they default pretty much out of the box. Setting fuzzy tolerance? Not a problem, it’s labeled as optional. The need to understand why you are performing analysis is not needed.
Now that isn’t to say Esri is doing something bad. They’re simplifying something that was very scientific and required understanding of FORTRAN or UNIX into something that almost anyone can do. I think at some level they should be commended for making GIS easier and not limited to a bunch of weirdos with Sun SPARCstation 20 workstations. But in doing so they turned something lightweight into something of a beast. Thus Brian’s heavy comment.
But that’s not the end to the story, at least from an Esri perspective. Esri at the same time they were throwing wizards in from of every tool in ArcGIS Desktop, created one of the most powerful GIS libraries ever created, ArcPy. It’s everything we wanted ArcInfo Workstation to become, a modern, no proprietary scripting language with tons of GIS analysis tools. But for some reason Esri doesn’t highlight it as they should. Just go to Esri.com and search for ArcPy. Typical Esri results, it’s a mess. Brian is reading this now nodding, “GIS is heavy”.
Heavy GIS is starting up ArcMap, starting up ArcCatalog, dragging and dropping into a wizard and fighting through the next screens. The process is similar in QGIS which seems to be adopting some of the same wizard dialogs as ArcGIS. They’re heavy because that’s what they need to be. Scott Morehouse years ago told me ArcGIS was complicated because it is “scientific software”. At the time I laughed but I do get it. It’s the long tail of long tails in GIS, solving GIS analysis in so many edge cases that it gets bloated.
Esri should3 have section of their website devoted to Python scripting. Showing how much easier (and faster) it is to do your analysis with ArcPy over ArcGIS Toolbox4. There are pieces all over their website about Python and ArcGIS, but “Scripting” section. That would go a long way to making Desktop GIS not heavy. Searching Google for “Esri Scripting” gives you a dead end to ArcScripts. That should change.
This hilarious article on abandoned Universities in Second Life got me thinking about 3D worlds and GIS.
Colleges were among those that bought the hype of the Linden Lab-developed virtual world. Many universities set up their own private islands to engage students; some even held classes within Second Life.
Most of these virtual universities are gone –– it costs almost $300 per month to host your own island –– but it turns out a handful remain as ghost towns. I decided to travel through several of the campuses, to see what’s happening in Second Life college-world in 2015
I mean seriously, what the heck were we thinking with Second Life? But while Second Life was more hype than function, KML and Google Earth was our great hope. KML export, Arc2Earth, SketchUp all were tools that were used liberally to export our GIS models to Google Earth in hopes we’d finally have a universal GIS viewer. My site is littered with KML export articles that we all thought would change our lives. But honestly none of them really have taken off. In fact I rarely create KML anymore, my clients just doesn’t use Google Earth anymore.
So where is Minecraft in all of this? Safe Software has an Minecraft conversation as part of FME. If I look at the analytics to this site, most of the top 10 search terms have some tie in to Minecraft. But it feels so much different from Second Life or even Google Earth. I honestly have put some thought into licensing FME and putting up a GIS to Minecraft conversion service due to the interest in it.
But are we exporting GIS to Minecraft for visualization? No but I think there is a different thing going on here. Minecraft is as consumer as we get. GIS is very enterprise and business focused. We’ve always wondered how do we get ordinary people to use GIS data1. What Minecraft does is bring all that analysis, data conversion, transformation and scripting into the mainstream. I can’t recall my son ever being so interested in anything as he is in Minecraft. I’m sure your kids are the same way.
I’m still not really sure if Minecraft GIS is has any more traction than Second Life or KML for GIS professionals. It could very well be that in 3 years I’ll look back on this post and laugh at my words. But I’m betting that Microsoft will make Minecraft bigger than it’s ever been and Minecraft export format will be built into every GIS package.
well other than to find a Starbucks↩
I had some endpoints of a lines that I needed to convert to GeoJSON today. Before I started I do what every GIS professional does, take inventory of the multitude of ways to actually accomplish this. I mentally jotted down the following:
- Esri ArcGIS
- Online tools (csv -> json)
I started to realize that these are all pretty heavy tools to just accomplish something as simple as a line string to line. We literally pull out a chainsaw when all we want it to trim a little piece of paper. Nothing simple about converting some coordinates into JSON. Enter Turf.js.
I posted this a while back on twitter but someone asked me about it this morning and I thought I’d share it here on the blog. The Visual Studio Blog has and article out on Why write Python in Visual Studio? which is worth reading for everyone writing Python on Windows.
Recently, Visual Studio 2015 was released with support for Python. Python Tools for Visual Studio (PTVS) are available to help throughout Visual Studio in all the places you’d expect, from editing and IntelliSense, to debugging, profiling, and publishing to Azure. You can find all the details and some video walkthroughs, documentation, and other resources on visualstudio.com, and the post announcing Python Tools 2.1 and Python Tools 2.2 beta.
The post is a great read into the choices the Visual Studio team made on how to integrate IntelliSense with Python. Honestly when VS 2015 came out I’ve started using it for all Python development on Windows and when I switch over to my Mac I really miss the features. GIS users have embraced Python and having a real IDE to help them is a huge plus. In the past I’ve avoided IDEs where I didn’t need them but with Python projects getting larger and more important, it really makes sense to organize them better.
The sale was a milestone for Israel’s young but huge startup community: The first Israeli consumer-app company to be bought for over $1 billion. In an instant, the whole “Startup Nation” decided to quit aiming for fast exits and build billion-dollar companies instead.
When Google bought Waze we were all amazed they paid $1B. Not so much in that we didn’t think Waze was going to sell for $1B1 but that Google needed them. In the end it was simple for Waze:
What made Google pretty attractive for us that No. 1, the company stayed in Israel. No. 2, we remained with our mission, to help drivers avoid traffic jams.
Well and that $1B was pretty attractive too. I’m honestly not sure what is going to happen to Waze moving forward. I still use it daily on my commute. Waze is partnering with cities to improve traffic results and I know millions of others rely on it for better traffic results than Google Maps or Apple Maps. But that’s the kicker right? Questions that come to mind to me are:
- What’s the incentive to innovate beyond improving traffic results?
- What’s the status of the maps behind the application, are they being updated?
- Does Google plan to shut Waze down and “integrate” traffic into Google Maps?
- Is Waze just another example of supporting a proprietary map only to see it be pulled away from the community?
Google bought Waze over 2 years ago. We haven’t seen anything new from Waze beyond these “partnering” programs2. I’ll continue to use Waze for my commuting because it is such a time save but the end game of Waze is probably not benefiting me.