A Whirlwind Tour of Spatial Joins
0 likes•867 views
The document discusses the basics of data integration and spatial joins, highlighting how to combine datasets based on location and proximity using FME's tools. It emphasizes the flexibility of spatial joins through various transformers and their applications in data validation and proximity detections. The presentation provides practical insights into managing disparate data formats and automating workflows for enhanced data usability.
A Whirlwind Tour of Spatial Joins
- 1. Data Integration Basics: Spatial Joins
- 2. Meet the Presenters Mark Stoakes Nampreet Singh Dale Lutz Olivia Wasalski
- 3. Maximize the value of your data by combining information based on location & proximity
- 4. Merging and joining disparate datasets using a spatial join can be easy... FME can do the work for you.
- 5. Flash Demo
- 7. FME® Integration Platform Connect. Transform. Automate. FME Desktop FME Server FME Cloud Build & Run Data Workflows Automate Data Workflows Automate Data Workflows (Hosted)
- 8. Merge and Join ANY Type of Data
- 9. Data integration difficulties often result from multiple data formats. Just because you have different kinds of data, doesn’t mean they can’t be brought together.
- 10. Our Mission To help you maximize the value of your data.
- 11. Agenda ● The Basics of Spatial Joins ● Transformer Categories ● Demos: ○ Merging Attributes of Multiple Datasets ○ Transformer Comparisons
- 12. What is a Spatial Join? When data is merged through a spatial join, the attributes from each dataset are combined together based on a their proximity or spatial relationship and then output as a single fused dataset. As with attribute joins, you can have one-to-one or one-to-many relationship.
- 13. What are we looking at? A spatial join uses geographic proximity to combine attributes. It moves the columns from the join features to the target features. The spatial relationship can be: ● Any intersection between two features. ● Two features touch at any location. ● The join feature is within the target feature. ● Two features are within a fixed distance. ● The join feature is closest to the target.
- 14. Joining Attributes Spatial Join Tabular Join/Merge
- 15. How Spatial Joins are Being Used ● Spatial Data Validation ● Asset finding ● Cell User Billing
- 16. There are Many Different Ways to the Destination
- 17. Two Categories: Proximity and Spatial Relation Proximity Nearest & dearest Includes: ● NeighborFinder ● Matcher ● PointOnPointOverlayer Spatial Relationship Exact match Includes: ● FeatureReader ● PointOnAreaOverlayer ● SpatialFilter ● SpatialRelator
- 18. Third Category: Destructive Joins Not covered in this webinar are joins that also change the geometry. These include: ● Clipper ● LineOnLineOverlayer ● LineOnAreaOverlayer ● AreaOnAreaOverlayer ● PointOnLineOverlayer
- 19. Follow the Flow-Chart Inspect your data and ask yourself: 1) Proximity or Spatial Relation? 2) Distructive or non-distructive join? 3) Simple or Sophisticated? Knowledge Base Article Merging or Joining Spatial Data
- 20. Nearest & Dearest with the NeighborFinder Proximity/Nearest Neighbor What’s closest to me? ● Closest washrooms to Skytrain Station ● Cell tower to cell tower proximity (handoffs during outages)
- 22. NeighborFinder Join text with leader lines on CAD Manhole features 1. NeighborFinder #1: attach text to the closest leader line 2. NeighborFinder #2: find the line feature closest to each Manhole feature More applications...
- 23. NeighborFinder 1. Assets on Road Network 2. Utility Network Migration More applications... 3. Linear Referencing 4. Cell tower to cell tower proximity: signal handoff during outages Webinar | June 1st Linear Referencing (LRS): How FME Measures Up
- 24. NeighborFinder ● To find all matches, Set Number of Neighbors to Find to 0 or blank ● Set Maximum Distance if you are looking for all or multiple matches within x distance of a feature. ● To accumulate attributes from multiple matches, enable Generate List (saved as list attributes visible in the Feature Information Window).
- 25. Matcher ● Detects proximity and exact match ● Can use geometry as the join key ● Option to compare attributes for differences Matched and/or Duplicates
- 26. Demo Matcher
- 27. Matcher ● Vector Tolerance: Fréchet distance (dog leash) method ● Generate List: merged attributes appear as list attributes (view in Feature Information Window) ● Use Match ID Attribute to perform joins downstream ● Also match 3D features, measures and geometry traits ● If matching on attributes only (not geometry), consider using FeatureMerger
- 28. Midstream Join with FeatureReader ● Simple midstream option ● Join against any database or spatial format ● Typically used for spatial joins but can also include a WHERE for tabular joins with database tables Let-your-database-do-the-work
- 29. Midstream Join with FeatureReader 1. Set the WHERE Clause to: “<Incoming Attribute>” = ‘@Value(<Initiator Attribute>)’ 2. Select the Spatial Filter “Initiator OGC- <predicate> Results” 3. Set Accumulation Mode to Merge Initiator and Result Documentation
- 31. SpatialFilter and SpatialRelator Comprehensive spatial relationships. Support OGC spatial predicates: ● Intersects ● Equals ● Touches ● Contains ● Within ● Crosses ● Overlaps ● Disjoint ● DE-9IM Merge Attributes
- 32. SpatialFilter ● Filters point, line and area features based on spatial relationships. ○ Pass ○ Fail SpatialRelator ● Determines what type of spatial relationships exist between sets of point, line and area features. ● Returns the DE-9IM representing the spatial relationship. ● Returns a list of all the candidates that match.
- 35. SpatialRelator: Refine spatial joins with DE-9IM TOUCHES predicate doesn't quite give us the result we want. But SpatialRelator returns the the 'actual' relation as a DE-9IM: This is for the hedges we want to filter out. _relationships{0}.de9im F1FF0F212 _relationships{0}.Legend 25 _relationships{0}.pass{0} TOUCHES Merge Attributes Note: You have to transpose the DE-9IM for SpatialFilter Jane Mason Suffolk Biodiversity Information Service Ipswich Museum, UK
- 37. Natural Resources Canada Feature catalog for Canada’s national mapping FME within NRCan A=Base B=Candidate FME Spatial Relations Defined
- 39. Automating Automate any of your workflows with apps on FME Server. Resources: Getting Started with FME Server Getting Started with FME Server Workspace Apps Getting Started with Gallery Apps Webinar - Creating No Code Web Apps with FME Server
- 40. Summary
- 41. By choosing the right Transformer, you can merge and join spatial data more efficiently.
- 42. Summary Spatial joins: needed for when you want to merge datasets but don’t have a common attribute ID. Lot’s of flexibility in FME: ● Simple joins - PointOnAreaOverlayer ● Proximity - NeighborFinder ● Comprehensive spatial relationships - SpatialFilter & SpatialRelator ● Let-the-database-do-your-work - FeatureReader Data used in the demos are from the City of Surrey Open Data Site, City of Vancouver Open Data Portal and UK Ordnance Survey Topography
- 43. Additional Resources Knowledge Base Merging or Joining Spatial Data Point in Polygon Calculations with the PointOnAreaOverlayer FME User Conference FME within Natural Resources Canada
- 44. Get Started Use a Transformer to join your data. Automate the process so you can focus on other tasks. Free Trial | Upgrade Gather your data.
- 45. Perpetual License Cloud Deployment Licensing Options ● Includes permanent license + annual maintenance ● Designed for smaller deployments ● Unlock enterprise automation without the hassle of hardware ● Flexible pricing options including hourly and annual subscription More info: safe.com/pricing Enterprise Subscription ● Subscription program ● Ideal for cross-department deployments
- 46. Thank you! Get a free trial of FME at safe.com/trial Connect with us for more FME
- 48. How to use this deck This deck follows a “Story” structure. It’s set up to draw the audience in and hold their attention. To give your most effective presentation, follow the suggested format and read the tips added to the Speaker Notes section. That being said, make this deck your own! There are TONS of slide options for you to use. You can find them when you add a new slide. →
- 49. Meet the Presenters Presenter’s Name Title Presenter’s Name Title Presenter’s Name Title
- 50. Agenda What we’re covering today. ● Add ● Bullet ● Points ● Here
- 51. Data integration makes your data more valuable. Use this statement slide to invite the customer into the “story.” Do this by identifying what they want (as it relates to the webinar)
- 52. Offer more / define the topic This is a good place to explain a process or term if you’re going to use it throughout the webinar. For example: “Businesses have lots of different kinds of data and multiple data management systems. Data integration combines various data types and formats into a single location – commonly referred to as a data warehouse.” Pro tip: Writing clear and simple is hard! Challenge yourself to use language that is easy to understand.
- 53. Offer more / define the topic For example: Data integration can help to: ● Utilize all the data that is encapsulated within your organization ● Break down data silos in favour of cross-functionality ● Create accessibility within your company
- 54. But data is messy. It’s scattered all over, it’s full of errors, it’s in different formats, it’s trapped in old programs. We both know what they want. But they’re having a problem getting it. They need help and that’s why they are here.
- 55. Talk about the external problem Spend some time talking about their problem. It helps to do it in this way, the external problem, the internal problem, and the philosophical problem. Examples are structured across these 3 slides. Data is not always easy to work with. The most common frustrations people have with their data are: ● Data that is not valid ● Data is in unusable formats ● Data processes are complex and it makes it is hard to collect or share data
- 56. Talk about the internal problem So a lot people just don’t use their data. What happens to many businesses, is they let their valuable data go wasted because it’s too frustrating or confusing to work with. If you’ve ever felt this way about your data, you’re in the right place. Talking about the problem is this way is important because it’s actually what people buy solutions to. They buy solutions to internal problems, not external ones.
- 57. Talk about why this is just plain wrong We can make better decisions if we have access to all the data. Businesses would be better off if they were able to easily access quality data. Businesses that make the best decisions are able to help the most people and ultimately, win the marketplace.
- 58. Our Mission To help you maximize the value of your data.
- 59. Company Profile Safe Software Years of solving data challenges 25+ 10,000+ Organizations trusting us worldwide Partners supporting our network 150+ 128 Countries with FME customers
- 60. FME® Integration Platform Connect. Transform. Automate. FME Desktop FME Server FME Cloud Build & Run Data Workflows Automate Data Workflows (on-premises) Automate Data Workflows (cloud) Get a free trial of FME Desktop and FME Server at safe.com
- 62. Section 1 Now you’re into the bulk of the presentation. Each section should offer the attendee a clear plan that they will follow after the webinar, or when they come across this problem again.
- 63. Connect Data between 450+ systems Extend Capabilities with custom connections, transformers, R & Python Run Workflows locally or publish to FME Server FME Desktop Quickly author repeatable, self-documenting data integration workflows using an intuitive visual interface. No coding required.
- 64. Pre-demo slide The Goal What is this demo trying to achieve? The Obstacles What is getting in the way? Why don’t we have the goal already? The Solution How are you going overcome the obstacles to accomplish the goal? (Hint: usually by using FME) The Result The goal was achieved. But why is that so significant? How is life better now?
- 65. Demo
- 66. Lesson Slide After giving a demo, having a slide to explain a lesson, or share some tips and tricks can be helpful.
- 67. Q&A
- 68. Section 2
- 69. Tips & Tricks Example of how you might share info.
- 70. Resources Place a transactional action slide somewhere in the middle of the webinar.
- 71. Section 3
- 72. Teach all the things!
- 73. Don’t waste the value of your data by keeping it stuck.
- 74. Summary Points If there are any key takeaways from the webinar you can share them here.
- 75. You can achieve data integration. Cast a vision of success for the customer. You can be more productive and focus on the work that matters when your data is validated, clean, and where it needs to be.
- 76. Call them to action Download FME 2020.1 Download the workspaces from this webinar Apply it to your own data to manage changes Free Trial | Upgrade
- 77. Perpetual License Cloud Deployment Licensing Options ● Includes permanent license + annual maintenance ● Designed for smaller deployments ● Unlock enterprise automation without the hassle of hardware ● Flexible pricing options including hourly and annual subscription More info: safe.com/pricing Subscription Licenses ● Subscription program ● Ideal for cross-department deployments
- 78. Build your own slides Graphic Elements
- 79. Place your image here
- 80. Place your image here
- 81. Place your image here
- 82. Place your image here Customer Story Add text Title
- 83. Build Schedules, applications and automations using a visual interface Scale To process massive amounts of data Eliminate The effects of data gravity by deploying engines close to the source FME Server Scale up processing power on-prem or in the cloud, and enable event-driven automations using a web user interface.
- 84. Share Connect data between 450+ sources Extend Extend FME’s capabilities with custom connections Enable Power business decisions by removing data silos Connect Your Data Sources FME supports geospatial data, structured and unstructured data, linked data, and time series.
- 85. Modify Restructure, filter, calculate and more Validate Validate to ensure high quality data Adjust Adjust workflows to meet changing requirements Transform Your Data Use any combination of transformers to address all levels of complexity.
- 86. Trigger Automate data integration using event-based workflows Assemble Easily build automations using a visual interface Eliminate Eliminate the manual effort of complex and repetitive tasks Automate Your Workflows Automatically provide integrated data to stakeholders on a real-time or scheduled basis.