Service Onboarding

Service Onboarding

This document contains instructions for onboarding your service based on either of the 2 authentication methods we support:

Onboard a new CloudAuth enabled service to FireFly

Prerequisites

  1. Get the CloudAuth and VPC endpoints for the service. These endpoints will be in one of the following places:

  2. Confirm that the service has AAA relationship with FireFly, and if not, request it from the current Devex-API team on-call for the API’s you want to integrate.

Modify the CDK package

The below steps are not required if your service is already on-boarded to FireFly and need to integrate a new API with that service.

  1. Create a new local dev branch.

  2. Add the CloudAuth and VPC endpoints to the CDK service package in Firefly.

  3. Add the service name in the ServiceType. For example:

    type ServiceType =
    | 'MusicIdentity'
    | 'MusicLimitManagementService'
    | 'MusicPatrol'
    | 'MusicPlaylist';

  4. Add the VPC endpoints of the service you are trying to integrate. For example:

    const VPC_ENDPOINTS: ServiceConfig = {
    MusicIdentity: {
        prod: {
            'us-east-1': 'com.amazonaws.vpce.us-east-1.vpce-svc-0f0c8b581a011fce4',
            'us-west-2': 'com.amazonaws.vpce.us-west-2.vpce-svc-0660c5ef17563aedb',
            'eu-west-1': 'com.amazonaws.vpce.eu-west-1.vpce-svc-0ff42dec38f343970',
        },
    },
}

    Note: For some services, if they don’t match the availability zones (az) of Firefly, you will need to override the az region.

  5. Add the service CloudAuth endpoints you are trying to integrate. For example:

    const ALIAS_RECORDS: ServiceConfig = {
    MusicIdentity: {
        prod: {
            'us-east-1': 'mis-q1t-vui-na-p-tcp.iad.amazon.com',
            'us-west-2': 'mis-q1t-vui-jp-p-tcp.pdx.amazon.com',
            'eu-west-1': 'mis-q1t-vui-eu-p-tcp.dub.amazon.com',
        },
    }
}

  6. Create a CloudAuth binding between the VPC and the endpoint that was defined earlier.

    • Create the cloudbinding with the appropriate type of hosted region. For example, amazon, hex, musicNova.

    • Below is the example for AmazonHostedZone.

    const musicIdentity = this.createCloudAuthBinding(vpc, amazonHostedZone, 'MusicIdentity', stage, region);

    • For hex hosted zone:

    const musicProduct = this.createCloudAuthBinding(vpc, hexHostedZone, '<service_name>', stage, region);

    • For musicNova hosted zone:

    const explicitSignalStorage = this.createCloudAuthBinding(vpc, musicNovaHostedZone, '<service_name>', stage, region);

  7. Now you’re ready to create a merge request. Note: If the service being integrated is in NAWS, then Firefly AWS accounts should be added in ‘allowPrinicpals’ section of service’s VPC endpoints to allow FireFly stack find service VPC endpoints. See this example.

Set up new service package

  1. Once the CDK MR is approved and the changes are deployed, request that maintainers create a client package as FF_servicename in Firefly clients.

    • Set visibility to internal, and initialize it with a README so it comes with a main branch.

    • Modify settings to squash commits and use fast-forward merges.

  2. Run yarn init, and commit based on the submission guidelines.

  3. Add the code for making requests to your service. Use this as an example: https://gitlab.aws.dev/amazonmusic/musicfirefly/clients/ff_musicidentityservice. Basic requirements include:

    • Package configuration files (package.json, yarn.lock, .gitignore, etc.)

    • Type definitions for the requests, responses and primitives.

    • A class that performs the request to your service. For example, MusicIdentityService.ts.

  4. Build the service package:

    yarn build

  5. To test your new service package, use yarn pack with your developer stack as described here.

Tip: You can use the steps in this quip document if you want to use coral to typescript model generator.

Changes to amu_webapi

  1. Add the dependency package in amu_webapi in package.json as described in the Developing against a deployed developer stack guide.

    @amzn/firefly-music-identity-service": "file:../ff_musicIdentityService/Firefly-Music-Identity-Service.tgz",

    Alternately, if your service is already deployed you can pull it from the published npm repo using:

    "@amzn/firefly-music-identity-service": "^1.0.1",

    Note: The caret symbol allows the package manager to update the dependency to the latest version that is backward-compatible with the specified version. It allows FireFly to automatically take in the latest minor or patch version update. For example: from 1.0.1 to 1.0.2, or 1.0.1 to 1.1.0.

    However, for a major version update such as from 1.0.1 to 2.0.0, run:

    yarn upgrade <package-name> --latest

  2. Add the endpoints added to the CDK on EnvUtil.ts

    // Register your service:
export enum ServiceClientSet {
    MUSIC_IDENTITY_SERVICE = 'MUSIC_IDENTITY_SERVICE',
}

// Add your endpoints:
export const serviceEndpointConfig: ServiceEndpointConfig = {

    MUSIC_IDENTITY_SERVICE: {
        prod: {
            'us-east-1': 'mis-q1t-vui-na-p-tcp.iad.amazon.com',
            'us-west-2': 'mis-q1t-vui-jp-p-tcp.pdx.amazon.com',
            'eu-west-1': 'mis-q1t-vui-eu-p-tcp.dub.amazon.com',
        },
    },
};

  3. Add service package config in ServiceConfig.ts:

    import { MusicIdentityService } from '@amzn/firefly-music-identity-service';

@Service()
export class MusicIdentityServiceConfig {
    'us-east-1' = new MusicIdentityService({
        region: 'us-east-1',
        stage: process.env.STAGE,
        timeout: SERVICE_TIMEOUT,
    });
    'us-west-2' = new MusicIdentityService({
        region: 'us-west-2',
        stage: process.env.STAGE,
        timeout: SERVICE_TIMEOUT,
    });
    'eu-west-1' = new MusicIdentityService({
        region: 'eu-west-1',
        stage: process.env.STAGE,
        timeout: SERVICE_TIMEOUT,
    });
}

  4. In the Constants.ts define the service and API constants:

    export const MUSIC_IDENTITY_SERVICE_SERVICENAME = 'MusicIdentityService';
export const MUSIC_IDENTITY_SERVICE_OPERATION_GET_AUTHENTICATE_PROFILE = 'AuthenticateProfile';
export const MUSIC_IDENTITY_SERVICE_OPERATION_GET_MUSIC_REQUEST_IDENTITY_CONTEXT = 'GetMusicRequestIdentityContext';

  5. Create a CloudAuth token map in CloudAuth.ts:

    • Import the constants we created in step 9:

    import {
  MUSIC_IDENTITY_SERVICE_OPERATION_GET_AUTHENTICATION_PROFILE,
  MUSIC_IDENTITY_SERVICE_OPERATION_GET_MUSIC_REQUEST_IDENTITY_CONTEXT,
  MUSIC_IDENTITY_SERVICE_SERVICENAME,
} from '../models/Constants';

    • Create a token map under the CloudAuthTokenMap interface

    export interface CloudAuthTokenMap {
MusicIdentityService: {
    GetMusicRequestIdentityContext: CloudAuthToken;
    AuthenticateProfile: CloudAuthToken;
  };
}

    • Make changes to retrieveCloudAuthTokenMap function in CloudAuth.ts by adding the service and API. This will generate and store the CloudAuthTokens once for all Integrations then pass through the Resolver chain in GraphQL to avoid multiple hits against CloudAuthTokenService. For example:

    this.getToken(
   MUSIC_IDENTITY_SERVICE_SERVICENAME,
   MUSIC_IDENTITY_SERVICE_OPERATION_GET_MUSIC_REQUEST_IDENTITY_CONTEXT,
   EnvUtil.getEndpoint(ServiceClientSet.MUSIC_IDENTITY_SERVICE),
   invalidateCache,
);

    • Create a map array. This will tag service with the cloudauthtoken

    const [
    MusicIdentityService_GetMusicRequestIdentityContext,
    MusicIdentityService_AuthenticateProfile,
] = await Promise.all(authPromiseArray);

const caTokenMap = {
    MusicIdentityService: {
        GetMusicRequestIdentityContext: MusicIdentityService_GetMusicRequestIdentityContext,
        AuthenticateProfile: MusicIdentityService_AuthenticateProfile,
    },
};

  6. Test if a connection is established from FireFly to the package we are integrating.

    • Create the types and check if we are able to retrieve the cloudauth. For example:

    import 'reflect-metadata';

import { DEP_WARN_RECOMMENDATION_TITLE, EntityType } from '../Constants';
import { Entity } from './Entity';
import { EntityConnection } from './connections/EntityConnection';
import { Field, ID, ObjectType } from 'type-graphql';
import { clientTokenType } from 'aws-sdk/clients/sts';

@ObjectType()
export class Identity extends Entity {
    constructor(self?: Identity) {
        super();
        Object.assign(this, self);
    }
    @Field({ description: 'cloudauth token.' })
    token: string;
}

    • Create a resolver. The following is an example of how to test and see the logs.

    import 'reflect-metadata';
import { Args, ArgsType, Ctx, Field, Query, Resolver } from 'type-graphql';
import { getLogger } from '../utils/LoggerUtil';
import { Service } from 'typedi';
import { GraphQLContext } from '../models/GraphQLContext';
import { Identity } from '../models/types/Identity';
const logger = getLogger({ name: 'Authorizer.ts' });
import { MusicIdentityService, GetMusicRequestIdentityContextRequest } from '@amzn/firefly-music-identity-service'
@ArgsType()
export class IdentityArgs {

}

export interface config {
    region: string;
    stage: string;
    timeout?: number;
}
const config: config = {
    region: 'us-east-1',
    stage: 'prod'
}

@Service()
@Resolver(Identity)
export class IdentityResolver {

    @Query(returns => Identity)
    async getMRICIdentity(
        @Args() {  }: IdentityArgs,
        @Ctx() ctx: GraphQLContext,
    ) {
        const identity = new Identity();
        identity.token = ctx.context.cloudAuthTokens!.MusicIdentityService.GetMusicRequestIdentityContext.token;
        const misService = new MusicIdentityService(config);
        const request: GetMusicRequestIdentityContextRequest = {
                activeCid: 'A1KE8TNGMC0JHH',
                activePid: 'amzn1.actor.person.oid.A27AEBVNNGB7MX',
                deviceId: '13436023902051611',
                deviceType: 'A16ZV8BU3SN1N3',
        };
        logger.error('the context is ', ctx.context);
        const response = await misService.AuthenticateProfile(request, ctx.context.cloudAuthTokens.MusicIdentityService.GetMusicRequestIdentityContext);
        logger.error('the response is ', response);
        logger.error('the token is ', ctx.context.cloudAuthTokens?.MusicIdentityService.GetMusicRequestIdentityContext.token);
        return ctx.context.cloudAuthTokens?.MusicIdentityService.GetMusicRequestIdentityContext.token;
    }
}

  7. Define the schema for graphQL in GraphQLSchema.ts.

  8. Deploy the changes as dev:

    yarn deployNA

  9. Proceed to Testing in Insomnia.

  10. After you get a successful response from the service, next create the actual models, resolver (example), DAO and transformers.

Test in Insomnia

  1. Open Insomnia and hit the endpoint with GraphQL query. For example:

query {
    getMRICIdentity{
        id
    }
}

If successful, you should see id as null in the Insomnia response.

  1. Check the logs in cloudwatch under your alias GraphQL.

  2. You should see your API response in the logs with the prefix [AXIOS]

Common mistakes

For troubleshooting service integration issues, see the Troubleshooting Guide.

Enable a new API for an existing service integration using CloudAuth

  1. Go to existing service-to-service relationship. For example, https://aaa.amazon.com/relationships/amzn1.aaa.rel.zwdu7oloksg4zofk2agq

  2. Under actions, click the checkbox for “I have read the instructions.”

  3. Click Edit Relationship.

  4. Make the desired changes, using add/remove.

  5. Click update request reply relationship.

  6. The pop-up will show instructions for consensus review.

  7. Under step 1, click create review. This will create the consensus review. Please share the URL with an authorized FireFly engineer for approval.

  8. Once approvals have been met, repeat above to create the same changes, but provide the consensus review ID under step 3 and click Update Request reply relationship.

  9. Add the new function in your service package (ex: FF_MusicIdentityService), and publish a commit using the feat: commit message to increment the package version.

    • Pull this new version into amu_webapi by updating the dependency’s version in package.json, and run yarn install. You are then free to use it within the FireFly code.

Add an IAM enabled service to FireFly

  1. Confirm that the service you are trying to connect to allows access via IAM.

  2. Create an IAM role in the service Firefly will call in prod, which enables API Gateway execute-api and gives trust relationship to the corresponding Firefly prod AWS account ID. Navigate to IAM in the AWS console for the service FireFly is calling.

    Note: FireFly only supports prod service integrations.

    AmazonAPIGatewayInvokeFullAccess
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "execute-api:Invoke",
                "execute-api:ManageConnections"
            ],
            "Resource": "arn:aws:execute-api:*:*:*"
        }
    ]
}

  3. Click on roles and create role. Set the trusted entity type as AWS account and mention the FireFly account id.

  4. On the next page, select the policies required to hit the API gateway endpoint.

  5. Add the IAM role to the Resource Policy for the specific endpoint on your service’s API Gateway.

    {
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::050160841055:role/MusicActivityFeedLambda-LambdaRole-13GC7XR8J6G30"
    },
    "Action": "execute-api:Invoke",
    "Resource": "arn:aws:execute-api:us-west-2:050160841055:*/*/GET/internal/feed/item"
},

  6. Copy the FireFly client code from other FireFly clients (i.e. FF_ActivityFeed) to make an HTTP request using the IAM auth by assuming the created role via STS.

    • Pass in the IAM Role’s ARN from the amu_webapi config in serverless.yml and ServiceConfig.ts

  7. Create a test file - for example, ActivityFeed.test.ts - in amu_webapi to validate your integration before adding GraphQL resolvers to expose the integration to end users.