Syncing AWS AppSync
AWS AppSync is a serverless GraphQL implementation released in July 2018. For those not familiar with GraphQL, it is an architecture that allows for seamless and flexible delivery of back-end data to any consuming or producing client application, although it is predominantly used to deliver data to a single-page application (SPA) such as React.js or Vue.js.
AppSync/GraphQL is an alternative to standard RESTful web services for delivering data to end-user applications, or to receiving changes from the same. Having worked with both, I believe that AppSync is a far better method to do this; less code, faster turn time, and fewer infrastructure requirements.
The CI/CD Challenge
However, syncing an AppSync deployment via Continuous Integration/Continuous Deployment is challenging. AWS provides the Amplify Console to assist with this, but the Console is very developer-focused. It manages ALL of the resources required to deploy a web application, including data and file storage (it will actually create your databases for you) and the GraphQL schema and resolvers.
While this is great for a developer and could be adopted into an overall DevOps approach to managing AppSync, we rejected this method for a number of reasons:
We greatly dislike “magic” buttons — things that cause multiple, sometimes hidden resources to be created in our infrastructures. If we don’t understand what’s being done, it makes fixing it when it breaks (which it will) that much harder. Amplify Console is a “magic” button.
Amplify actually creates/changes the DynamoDB tables for you. While this is awesome for the front-end application, often (always?) there are back-end processing dependencies on those tables that are not reflected in the Amplify configuration. You can imagine the chaos that often ensues.
Amplify doesn’t handle the AppSync extensibility into Lambda functions well. Simply put, if you want to do something custom with a Lambda function inside of AppSync, you’re outside of the Amplify sandbox. You might be surprised how often you will want to do this. For example, we often use a GraphQL mutation to initiate an AWS StepFunctions execution — something that is not AppSync native.
If you decompose AppSync, you’ll find that it consists of five different components: the GraphQL API endpoints, Data sources (DynamoDB, Aurora, Lambda, etc.), GraphQL Schemas, Resolvers and Functions.
AWS Resource CI/CD
The GraphQL API Endpoints and the Data sources are AWS infrastructure resources. We chose to create and manage these resources using Terraform. This allowed us to control their creation and change cycle discretely, just like we control all of our other AWS resources. It also allows us to easily manage the integration of those resources (e.g. – DynamoDB tables) into back-end Lambda or microservices processing. For example, we often set up DynamoDB Streams connected to Lambda functions for data-driven back-end processing.
The GraphQL Schema, Resolvers and Functions are a different matter. The Schema is written in the GraphQL language, while the Resolvers and Functions are written in Apache Velocity Template Language (VTL). These components are loosely coupled in AppSync with the API and the Data sources. As such, they may be changed very dynamically without directly impacting other AWS resources. Basically, they are code artifacts and, we believe, should be managed as such.
For each of these artifacts we create a git repository that follows the Gitflow branching workflow. To each repository we add an automated build that posts code changes to the correct AWS AppSync environment based upon git branch (develop -> dev, release/* -> stage, master -> prod). For each type of code artifact, we use an open-source python application to accomplish the post to AppSync. These applications are listed below, with links to their PyPI page:
Our solution allows for the full CI/CD integration of AppSync into a system that consists of more than just a SPA front-end, while allowing DevOps to maintain the infrastructure and Developers to maintain the actual API. We believe this is superior to the CI/CD process supported by Amplify Console — which places certain resources out of the control of DevOps and “hides” them from other system provisioning and management tools (e.g. – Terraform).
We’d be happy to hear your thoughts on the above, or your methods for solving CI/CD with AppSync.
Feedback? contact Joe with any comments
About the Author
An experienced C-Level executive with a breadth of experience in operations, sales, marketing, technology, and product development. Joe specializes in innovation and change leadership, having had success as both an entrepreneur and intrapreneur, across numerous industries including defense, insurance, telecommunications, information services and healthcare industries.
Joe relishes the opportunity to make a real difference, both in the success of the corporation and in the lives of the people he interacts with. A self-starter that neither requires nor desires large amounts of oversight having succesfully built numerous DevOps teams and Solution Architected some of the most advanced and secure utility computing applications.