Using AWS Lambda* and Feed Me to Create an Automated Content Pipeline

Transcript

1. JUAN J. RAMIREZ (@WHOISJUAN) USING AWS LAMBDA* AND FEED ME

   TO CREATE AN AUTOMATED CONTENT PIPELINE. DotAll Conf | September 2019 | Montreal, CANADA.

2. AGENDA - Introduction & Logistics - What is This Presentation

   About? - What's Feed Me? - What's Waveguide and How It Uses Feed Me? - Lambda (Serverless) Middleware. - Use Cases - Q&A

3. INTRODUCTION Juan J. Ramirez / JJ / @whoisjuan - Originally

   from Colombia. - 8.5 years as a UX Designer and Product Professional...And Developer Just for Fun. - Worked on several startups. Most recent experience Procore Technologies, Amazon (AWS) and currently GitLab. A lot of consulting too. - A lot of side projects. Currently building Waveguide (A Design Knowledge Base) on top of Craft CMS. 2 years. Craft 2 ->3

4. LOGISTICS Super Pressing Questions? Just raise your hand and ask.

   Otherwise save it for Q&A. Juan's Accent (Not hard to pick but...) If you didn't understand something I said. Please Tell Me! Technical Presentation Yes. But Non-Techies Welcome (Friendly Format).

5. WHAT IS THIS PRESENTATION ABOUT? Understanding Feed Me We will

   understand Feed Me and what offers besides what you see. The Power of Feed Me + Serverless We will understand how to combine a Serverless API with Feed Me to push content to Craft CMS. Examples and Ideas I will show you the Waveguide Example, but also explore other potential uses cases and workflows.

6. Feed Me is a Craft plugin for super-simple importing of

   content, either once-off or at regular intervals. With support for XML, RSS, ATOM, CSV or JSON feeds, you'll be able to import your content as Entries, Categories, Craft Commerce Products (and variants), and more. Source: https://docs.craftcms.com/feed-me/v4 WHAT'S FEED ME?

7. IMPORT - SETTING UP THE FEED SOURCE

8. IMPORT - SETTING UP THE FEED DESTINATION

9. IMPORT - SETTING UP THE STRATEGY

10. IMPORT - MAPPING TO FIELDS

11. LET'S THINK ABOUT THAT PROCESS AND THAT DESCRIPTION

12. Feed Me is a Craft plugin for super-simple importing of

    content, either once-off or at regular intervals. With support for XML, RSS, ATOM, CSV or JSON feeds, you'll be able to import your content as Entries, Categories, Craft Commerce Products (and variants), and more.

13. That sounds a lot like an API, doesn't it? We

    are consuming structured data, like JSON, and pushing it to Craft CMS via a console that allows us to map to certain fields (effectively mapping to the database).

14. But until this point, Feed Me sounds like a tool

    that allows you only to import content via a Console UI to manage migrations and bulk uploads of content. How can we automate anything if we don't have programatic access to it?

15. THE DIRECT FEED URL Every time we hit this URL,

    Feed Me will run that particular feed. We can also replace the default file by passing a url parameter with a new structured data file.

16. FEED ME CAN BE A PSEUDO RESTFUL API (FOR CREATE

    AND UPDATE OPERATIONS) Endpoint We have an endpoint that we can hit programatically and can receive some data via hosted files. Rules We can create rules on how the data is consumed by Craft (import strategies) Mappings We have mappings between the structured files we are giving Feed Me and Fields in Craft CMS.

17. PROS - Easy to Learn and Setup (Almost No Learning

    Curve / No PHP Required) - Intuitive way of creating mappings / UI Console. -Multiple Data Formats -Built-in Logging PROS & CONS OF USING FEED ME AS AN API FOR CONTENT CREATION CONS - Slow / Not an actual RESTful API. - Only suited for internal use / internal tools. - Only one GET endpoint (kind of hacky.)

18. FEED ME IN A NUTSHELL

19. FEED ME IN A NUTSHELL

20. That's cool and all but the name of the talk

    mentions Lambda/Serverless and Content Automation. Where's that?

21. WAVEGUIDE CASE STUDY Using Feed Me + A Static HTML/CSS/JS

    File + AWS Lambda to Capture Design Examples all over the Web.

22. WHAT'S WAVEGUIDE? Design Knowledge Base Capture Examples of Implemented Designs

    Built on Craft From The Beginning Leveraging the excellent content structuring within Craft. Multiple Use Cases - Volume Matters Waveguide it's only useful with content volume.

23. UPLOAD CONSOLE Created an Upload Console We submit a website

    URL and AWS Lambda and Feed Me takes care of the rest. AWS Lambda as Middleware Lambda receives the requests, takes the screenshots, then calls different machine learning APIs to extract meaning out of the website screenshot and then forms an object that gets pushed via Feed Me.

24. LIVE DEMO TIME LET'S CHECK IT OUT https://www.waveguide.io/upload-console-lp Of course

    this is protected under {% requireAdmin %}

25. WHAT CAN WE DO WITH A SERVERLESS MIDDLEWARE? Parse, "Massage"

    and Enrich Data A Serverless Middleware allows you to do multiple data manipulations before they reach Craft CMS. Control How and When To Push Data A Serverless Middleware allows you listen for events and create crazy rules on how to push and deliver content.

26. CAVEATS OF A SERVERLESS MIDDLEWARE + FEED ME IMPLEMENTATION Feed

    Me is Not An Actual RESTful API Because of this any data you prepare must be served through a static file in some cloud storage. Or using a hacky approach (Encoded JSON as a URL parameter) Serverless is Easy But It Requires a Different Mindset Serverless it's pretty different from writing a PHP script and spinning it on your own server. It's a fully isolated and stateless approach.

27. 1 Automatic Curation / Topic Selection Someone can create boilerplate

    articles from other content online The middleware can extract all the references from a submitted article. It can also query language APIs to extract core ideas. An author can use the draft created automatically to produce a new article on the same topic. USE CASES FOR A FEEDME + SERVERLESS IMPLEMENTATION

28. 2 Automated System of Record A middleware can be programmed

    to receive different data coming from sources like Slack or Google Docs and it can feed Craft CMS entries that can serve as an internal system of record for a company. You can setup this for a client without worrying about PHP or coming up with a crazy architecture to handle it. Your client can add more fields through time and Feed Me can scale as easily. USE CASES FOR A FEEDME + SERVERLESS IMPLEMENTATION

29. 3 IoT Data Capturing https://dotall.com/sessions/building-iot-solutions-using-core-cms- features - Watch this talk

    from Dot All 2018 Craft CMS can be a system of record for IoT solutions. With Feed Me and a Serverless Middleware you can create a way to capture field data on demand. For example using an Amazon IoT Dash Button to capture a Photo of a location + Weather Data + Other Data and then record it in Craft. You can build this without PHP, and by using simple JS code snippets. USE CASES FOR A FEEDME + SERVERLESS IMPLEMENTATION

30. 4 React to Any Event. Let Your Imagination Fly. With

    AWS Lambda, Azure Functions or Google Cloud Functions, you can react to hundreds of events to create content. If you want to take it further, you can use services like Zapier, IFTTT, Tray.io, etc; to build more complex workflows. You can create crazy things in less than a day. USE CASES FOR A FEEDME + SERVERLESS IMPLEMENTATION

31. In Craft CMS you can build a whole website with

    zero PHP knowledge. This architecture is almost as simple and opens a world of possibilities. You can increase the value you give to your customers exponentially by offering solutions based on these ideas / architecture. CONCLUSION: WHY WE SHOULD USE FEED ME AND SERVERLESS TO AUTOMATE CONTENT?

32. Q&A Contact Me @whoisjuan (Twitter, GitHub) www.whoisjuan.me (My portfolio) www.waveguide.io

    (Expand your Design Knowledge) getscreenshot.rasterwise.com (For your screenshot needs)