Learnings in the world of AWS Amplify

This is a growing collection of my learnings from building web applications with AWS Amplify

Image for post
Image for post

AWS Amplify is a wonderful idea. In theory it simplifies all the messy coordination of different Amazon Web Services and speeds up building new products. In practice however, Amplify is not quite there yet. Here I describe my learnings of how to use Amplify in practice and how to circumvent the inevitable problems with a new technology.

The paragraphs are in no particular order — I will add to this article each time I discover a solution to a non-obvious Amplify challenge. Feel free to come back frequently and learn from my journey. If you have tips or better solutions, please feel free to comment or reach out to me.

Here we go with a bunch of random paragraphs about my Amplify learnings:

Photo by Tim Mossholder on Unsplash

Table of Contents

  • Accessing GraphQL & AppSync from Lambda
  • AppSync with Cognito and IAM Access
  • Calling the REST or GraphQL API from your frontend with Amplify
  • Strange behavior of GraphQL list queries

Accessing GraphQL & AppSync from Lambda

In an ideal world, accessing an AppSync instance (created with Amplify) from a Lambda function (also created with Amplify) should be trivial. Unfortunately it is not (yet).

What doesn’t work

First, you should not use amplify-js on the server-side. It was build for the client-side and might fail in unexpected ways if used in a NodeJS Lambda function. While I have seen people make it work somehow, it feels like a ticking time bomb to me — and I like to avoid these.

Another option would be to access your AppSync source (most likely a DynamoDB) directly from Lambda. But with this your subscriptions on the client will not work: if you create an item directly in DynamoDB and your client is subscribed to receive new items, your client will not update. So accessing the DynamoDB directly is also not an option.

What works

The (currently as of Nov 2020) correct and recommended way to access AppSync data from your Lambda function is via https calls to the graphql API. The details are a little involved though, so your best chance is to copy/paste the code from the official Amplify documentation:

Get the AppSync GraphQL URL

The AppSync URL can be provided to your Lambda function by Amplify. If necessary, do an amplify update api and give your Lambda access to AppSync.

Allow access to the AppSync API from Lambda

Your AppSync API is protected by default. For public APIs it might be as simple as an API key (this is shown in the “simple” example in the Amplify docs). API keys are great to try things out during development, but for most production APIs, you don’t want them to be accessible to anybody with an API key (otherwise people could freely create or delete user profiles for example).

Most people have already set up AWS Cognito for authentication. A naive approach could be to manually create an “API” user in your Cognito user pool and give them full access to your API. While this certainly works, it is an ugly approach that can lead to problems down the line. So I discarded it.

The “correct” way to access one AWS service (like AppSync) from another AWS service (like Lambda) is by using IAM roles or users. Luckily, this is rather easy to set up with Amplify:

  1. Add IAM authentication to your Amplify application via amplify add auth and picking IAM.
  2. Add IAM auth to your AppSync API via amplify update api . Luckily you can have multiple different auth providers for the same AppSync API — I use Cognito and IAM in parallel: Cognito for managing access for my users and IAM for managing access of my server-side services.
  3. Explicitly add IAM auth to your AppSync schema (for details see the section on “AppSync with Cognito and IAM Access” in this article)

Conclusion

I hope that in the future Amplify will be able to connect Lambda and AppSync more seemlessly. Until then we have to make https calls that we manually sign with the correct AWS credentials.

AppSync with Cognito and IAM Access

Most production apps need to access their data from various sources: user clients (web, app, desktop), servers (Lambda, legacy applications, …) and maybe even third-party services (if you provide a public API). Luckily, Amplify allows us to access the same AppSync instance with different auth methods for each such source: we can use Cognito user pools, IAM users & roles or API keys. And we can use all of them in parallel!

So our web application can use:

  • A Cognito user pool to allow fine-grained access to only the portions of AppSync data the user is allowed to access (e.g. they can change their own user profile, but only read other user profiles)
  • IAM users to allow necessary access for our business logic on servers, Lambdas and so on.
  • An API key to allow third-party developers rate-limited access to public parts of our AppSync API

The pitfall

The tricky part comes when combining this with Amplify’s simplified AppSync schema! Most code examples assume that you’re directly manipulating the GraphQL schema, which is automatically generated (and regularly overridden) by Amplify. In the GraphQL schema you would simply use annotations like @aws_iam or @aws_cognito_user_poolsto protect certain resources via IAM or Cognito user pools. However, these do NOT work as expected in Amplify schemas!

For the simple case of having just one auth provider (e.g. only Cognito user pools), you can get away with simply adding the @auth annotation to your type definition:

type User @model @auth {

}

But once you use multiple auth providers in parallel (like IAM and Cognito user pools and API key), you need to use the correct Amplify (not GraphQL!) auth annotations to make it work. Here is how you could protect a User type for clients (with Cognito) and Lambdas (with IAM):

type User
@model
@auth(
rules: [
{ allow: owner }
{ allow: private, provider: userPools, operations: [read] }
{ allow: private, provider: iam, operations: [create, update] }
]) {…}

In this example I allow the currently authenticated user (i.e. the Cognito user signed in to my client app) full access to change and delete their own user profile. For all other client users I allow reading the user profile, but not manipulating it. Finally I allow creating and updating user profiles via an IAM user with the necessary rights (used e.g. by the Lambda method which creates user profiles after signup through Cognito).

So instead of using the GraphQL annotations @aws_iam or @aws_cognito_user_pools , we have to use the Amplify annotation @auth and pass the auth provider as a param.

Note that AppSync has the concept of a “default” auth provider. If you do not provide a provider params, AppSync will assume you want the default provider (which in my above example is userPools ).

It took me a little while to piece together how to do this within the Amplify schema. But now I (and you) know how to write auth annotations in Amplify, it is pretty easy and powerful to manage secure access from clients, servers and third-parties in parallel.

Calling the REST or GraphQL API from your frontend with Amplify

From your frontend webapp you certainly want to make calls to your REST API (e.g. a Lambda function with an express.js server). My initial reflex was to use axios or fetch for this. While I was trying to figure out how to authenticate with the Cognito user in such requests, I found a much simpler way to call the REST API…

Amplify provides a client-side API module in aws-amplify . You can make calls on this module as easily as calling API.post(<API_NAME>, <PATH>) ! The great part: this will automatically add the auth headers for the currently signed in Cognito user to the request — you don’t even have to worry about it.

This same API module can also be used for a GraphQL API. Simply call API.graphql(query: …, variables: …) and it will also add the current user’s auth headers.

By using the API module you don’t have to worry about the correct endpoint url or how to authenticate the current user. It is all done for you by Amplify.

Strange behavior of GraphQL list queries

When getting a list of items from your AppSync API, you can get unexpected behavior! This happens, if you use authentication on the model or if you use filters in the query.

I tried to fetch an item by matching one attribute and it returned nothing with limit=1 , but returned an item with limit not set (I believe it defaults to 10). Similar strange effects happen, if you want to paginate and retrieve 10 items of a list of the user’s items — you might only get back 7 items despite the user having 50 items in the DB.

Explanation

As explained on the Github issue, we have to understand how DynamoDB and AppSync work. Let’s assume we call listPosts with a filter: {title: {eq: "Hello"}} and limit: 1 :

  1. DynamoDB will fetch the first post from the DB. Let’s assume this first post has the title “First Post”.
  2. The result is then filtered for those with title “Hello”. This leads to an empty result, because “First Post” != “Hello”
  3. We (somewhat unexpectedly) get items: [] back from our query

A similar thing happens for authentication. Let’s assume listPosts without any filters or limits, but with the user only able to access their own posts:

  1. DynamoDB will fetch the first set of all available posts (10 by default, unless you explicitly set limit higher) from the DB. Let’s assume that 7 of these belong to our user.
  2. The result is then filtered down to those posts, which the user can see. This results in 7 posts.
  3. We receive 7 post items, even though the user might have hundreds of posts in the DB

All this was very unexpected to me and cost me hours of bug hunting. Luckily I found a solution on the Github issue.

The Solution

The solution is simply not to use list queries for retrieving items based on certain attributes! Rather use an index for each attribute which you might use to filter items by. This index can be easily created in Amplify by adding the @key to your schema.

If you want to find posts by title, you would add the following key to your post model in the schema:

type Post @key(name: “byTitle”, fields: [“titel”]) {…}

For the case of authentication, you could for example use Cognito user groups:

@key(name: "byGroup", fields: ["group"], queryField: "domainsByGroups")

It’s all pretty unexpected, if you were used to relation databases and SQL. It takes some time to understand that NoSQL DBs like DynamoDB are not good for making “complex” queries.

Written by

Agile Coach, Business Innovator, Software Engineer, Musician

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store