Develop and publish contract-first API to LKABs API Management platform

This guide shows how to develop and publish an API based on an OpenAPI Specification into LKABs API Management platform.

Release Cycle

Each API SHOULD be stored in a GIT repository with a dedicated release pipeline. The release process is based on 3 main branches that each points to one or many API Management environments. All work should be done using a feature branch that then is merged into the developed branch and then pulled downstream to the next branch.

Branch	Will be deplyed to
develop	development environment `apim-apimgmt-dev-weu-001-lkab`
release	test- and stage environments `apim-apimgmt-test-weu-001-lkab` and `apim-apimgmt-stage-weu-001-lkab`
master	production environment `apim-apimgmt-prod-weu-001-lkab`

When developing the API will pass 3 maturity steps.

Experimental → develop
Unstable → release
Stable → master/main

When the feature branch is pushed to the remote repository a build is started where the code is checked using a linters and a build is performed.

Skipping environments

It is often the case that the backend api doesn't have the corresponding environments. For example the backend only have a test and production. Then you probably just want to handle those two environments in API Management. This can be done by changing the generated azure-pipeline.yaml in your api repository.

SKIP_DEV_DEPLOYMENT: true|false
SKIP_TEST_DEPLOYMENT: true|false
SKIP_STAGE_DEPLOYMENT: true|false
SKIP_PROD_DEPLOYMENT: true|false

...

stages:
  - template: ./src/pipeline-templates/.azure-pipelines-api-deployment.yml@templates-apim
    parameters:
      DEPLOYMENT_NAME: $(Build.Repository.Name)
      MARKDOWN_LINT_ENABLED: ${{parameters.markdownlintEnabled}}
      OAS_LINT_ENABLED: ${{parameters.oaslintEnabled}}
      LOCATION: 'westeurope'  
      SKIP_DEV_DEPLOYMENT: false
      SKIP_TEST_DEPLOYMENT: false
      SKIP_STAGE_DEPLOYMENT: false
      SKIP_PROD_DEPLOYMENT: false

Branch strategies when skipping environments

Some common use cases and suggestions

Only production backend

  SKIP_DEV_DEPLOYMENT: true
  SKIP_TEST_DEPLOYMENT: true
  SKIP_STAGE_DEPLOYMENT: true
  SKIP_PROD_DEPLOYMENT: false

Only work with pull requests against master/main

Only test and production backend

  SKIP_DEV_DEPLOYMENT: false
  SKIP_TEST_DEPLOYMENT: true
  SKIP_STAGE_DEPLOYMENT: true
  SKIP_PROD_DEPLOYMENT: false

Only work with pull requests against develop during development and master/main for production.

Linter tools:

The code is then brought in from the feature branch making a pull request against the develop branch. To complete the pull request all checks need to be in success state.

The pull needs to have a work item.
The pull request needs to be reviewed and approved by at least one other team member in the Azure DevOps project.
If there are any comments those needs to be resolved.
Build must have succeeded.

Suggested tools to install

Firewalls

LKAB API Management platform will need access to the API backend that is to be exposed and requires firewall openings.

Azure APIs and other public APIs requires firewall openings in Azure Firewall.
On-premise API requires firewall openings in the On-premise data center firewall. See the Firewall guide here. Request for these openings as early as possible.

Develop and publish API

Step 1 - Reserve an API Id and basepath

Each API SHOULD have an unique API Id on the format api[0000-9999]-[api name], as well as an unique API basepath.

The API basepaths are globally unique within the API Management platform. To avoid collisions, the basepath assigned to an API published to the LKAB API Management platform, need to conform with the pre-defined set of basepaths in the Pre-defined basepaths document, and be unique within the API Catalog.
Assignement of API basepath is part of the onboarding process when a new API are about to be published to the API Management platform, agreed upon between API owner and the API Management team.

The agreed and assigned API basepath SHOULD be documented along with the assigned API Id in the API Catalog before going further in the deployment process.

Step 2 - Create work item

To collect information about api:s and track the progress of the onboarding process, all api:s should have a Product Backlog Item (PBI) on the api-management-apis taskboard. If a PBI already exist, continue with Step 3.

To create a new PBI:

Create a PBI via this PBI creation link
Update the title
Add work item 35358 New Apis as parent
Fill in the form in the description (can be edited later too)
Save
Add tasks via 1-Click Child-Links action

Some tasks refers back to this document, helping you completing the tasks.

Step 3 - Create repository

Follow the instructions in this section.

Step 4 - API Information

Replace the OpenAPI Specification in the [src/openapi] folder with the OpenAPI Specification for your API and ensure the OpenAPI Specification to be compliant with Stoplight spectral-cli linter.

The file SHOULD be named oas.yml.

NOTE! There is currently a size limitation on the oas file due to the usage of the function loadYamlContent, see Issue: Increase MaxLiteralCharacterLimit for loadYamlContent. Maybe it's possible to add the oas content inline in the bicep/api-definition.bicep file.

Common OpenAPI updates

These are some common updates that are needed to make the OpenAPI Specification valid.

Add tags, contact and change host name of url
Replace summary name with description on the operations
Change description for the response 200 to "Success"
Add empty tags element to operations that are missing this

openapi: 3.0.1
info:
  title: LKAB.Test.API
  version: '1.0'
  contact: # Add this section 
    name: <name of the technical contact>
    email: <email of the technical contact>
  description: LKAB Test API
servers:
- url: https://api-lkab.lkab.com/test/connectivity # Change host
tags: # add tags section and tags used on operations
- name: WeatherForecast
paths:
  "/WeatherForecast":
    get:
      tags:
      - WeatherForecast # If this tag exist, tag need to be added in tags section above
      operationId: GetWeatherForecast
      description: Get Weather Forecast # Summery to description if missing, or add a description
      responses:
        '200':
          description: Success # Add a valid description

Tip: run spectral lint **/openapi/*.* --ruleset ./.spectral.yml need npm i @stoplight/spectral-cli -g in the root of the repo

Step 5 - Update configuration information

Update the...

OpenAPI Specification in the repo folder src/openapi, this should have been done already in the previous step
pipeline id in the README.MD file located in the root folder of the repo. Replace TODO:addPipelineId with the pipeline id of the pipeline in Azure DevOps. Also, if you have a self-hosted repo (ie the repo is not located in the api-management-apis DevOps project), you need to update the URL path and replace the api-management-apis part with your project name.
OPTIONAL: Update api-definition.bicep located in the repo folder src/bicep/ file with policy if there is a need to have separate policy handling for operations. See section policies, below for more information about how to do this.
Parameter files located in the repo folder src/bicep/environments/[regions: westeurope] Note! At the moment we are just supporting westeurope, See Parameter files for mor information
Optional: Run command markdownlint **/*.md --config ./.markdownlint.yml locally to check that markdown is following markdown standard. Note! This requires that the Markdownlint-cli has been installed. Fix markdown errors if any.
Commit the changes and push to remote repository. git commit -am "Configure and publish api" and git push

Policies

More info

Parameter files

The parameter files for each environment need to be updated. Examples exist in the api-definition.dev.bicepparam locates in teh folder src/bicep/environments/[regions: westeurope]

Allowed Origins (OPTIONAL)

If origins need to be allowed uncomment the allowed Origins parameter.

// Add allowed origins if CORS is needed, if set a cors policy fragments is added
param allowedOrigins = [
  'foo.com' // Try to avoid *
]

Update backends

These are the backend security types that are currently supported out of the box.

Backend protected with:

Managed Identity, this is the recommended backend security to use when possible (Can be used with almost all Azure Native services)
Bearer token grant type Client Credentials
Bearer token grant type Password
Bearer token grant type Refresh Token
Behalf Of The Authenticated Identity, this is used when the JWT i checked in the API and then should be passed forward to the backend.
API Key, this SHOULD BE AVOIDED if possible, ask for another backend security type.
Bearer token grant type Client Credentials (Static token), this SHOULD BE AVOIDED if possible but sometimes we get a static bearer to use. This we should consider as bad as a Basic Authentication
Basic Authentication, this SHOULD BE AVOIDED if possible, ask for another backend security type.

Add backend that is needed by the API by uncomment the bicep fragment in the parameter file that match the correct backend security type.

NOTE! If more then one backend is use the name value MUST be set for the 2...n backends, the repoName part in apiPolicy.xml file located in the repo folder src/bicep/policies need to be changed to [repoName]-[backend-name]. If only one backend is used it is recommended to not include the name parameter in the backend.

Ex. One backend:

param bearerTokenGrantTypeClientCredentialsBackends =  [
  {
    endpointUrl: 'https://conferenceapi.azurewebsites.net'
    clientId: '11111111-1111-1111-1111-111111111111'
    #disable-next-line no-hardcoded-env-urls
    clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'  
    // NOTE! 
    // If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other. 
    // This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
    // then the tokenUrl should not be set
    #disable-next-line no-hardcoded-env-urls
    tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
    // Optional
    description: 'The api0000-demo-conference backend info'
    scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
  }
]

Ex. More then one: NOTE! The name in the second object

param bearerTokenGrantTypeClientCredentialsBackends =  [
  {
    endpointUrl: 'https://conferenceapi.azurewebsites.net'
    clientId: '11111111-1111-1111-1111-111111111111'
    #disable-next-line no-hardcoded-env-urls
    clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'  
    // NOTE! 
    // If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other. 
    // This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
    // then the tokenUrl should not be set
    #disable-next-line no-hardcoded-env-urls
    tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
    // Optional
    description: 'The api0000-demo-conference backend info'
    scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
  }
  {
    endpointUrl: 'https://conferenceapi.azurewebsites.net'
    clientId: '11111111-1111-1111-1111-111111111111'
    #disable-next-line no-hardcoded-env-urls
    clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'  
    // NOTE! 
    // If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other. 
    // This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
    // then the tokenUrl should not be set
    #disable-next-line no-hardcoded-env-urls
    tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
    // Optional
    description: 'The api0000-demo-conference backend info'
    scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
    name: 'api0000-demo-conference-backend-sessions'
  }  
]

Make similar changes in the parameter files for for test, stage and prod, where the information is related to each environment. Example on how to reference backends is found in the apiPolicy.xml file located in the repo folder src/bicep/policies.

Uncomment the backend policy fragments with the correct "grant type" in the apiPolicy.xml, uncommented policy fragment that is not needed can be removed.

Ex: Client Credentials, uncomment this two sections between 

        <!-- #### GRANT TYPE Client Credentials:  bearerTokenGrantTypeClientCredentialsBackends, Example

        <cache-lookup-value key="@("api0000-demo-conference-bearerToken")" variable-name="bearerToken" />
        <choose>
            <when condition="@(!context.Variables.ContainsKey("bearerToken"))">
                <send-request ignore-error="true" timeout="20" response-variable-name="accessTokenResult" mode="new">
                    <set-url>{{api0000-demo-conference-tokenUrl}}</set-url>
                    <set-method>POST</set-method>
                    <set-header name="Accept" exists-action="override">
                        <value>*/*</value>
                    </set-header>
                    <set-header name="Accept-Encoding" exists-action="override">
                        <value>gzip, deflate, br</value>
                    </set-header>
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/x-www-form-urlencoded</value>
                    </set-header>
                    <set-body>@{
                                return "client_id={{api0000-demo-conference-client-id}}&client_secret={{api0000-demo-conference-client-secret}}&scope={{api0000-demo-conference-scope}}&grant_type=client_credentials";
                        }</set-body>
                </send-request>
                <set-variable name="accessToken" value="@(((IResponse)context.Variables["accessTokenResult"]).Body.As<JObject>())" />
                <set-variable name="bearerToken" value="@((string)((JObject)context.Variables["accessToken"])["access_token"])" />
                <set-variable name="tokenDurationSeconds" value="@((int)((JObject)context.Variables["accessToken"])["expires_in"])" />
                <cache-store-value key="api0000-demo-conference-bearerToken" value="@((string)context.Variables["bearerToken"])" duration="@((int)context.Variables["tokenDurationSeconds"])" />
            </when>
        </choose>
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["bearerToken"])</value>
        </set-header>
        -->

and

        <!-- 
             #### GRANT TYPE: 
                bearerTokenGrantTypeClientCredentialsBackends, Example
                bearerTokenGrantTypePasswordBackends, Example
                bearerTokenGrantTypeRefreshTokenBackends, Example

        <choose>
            <when condition="@(context.Response.StatusCode == 401 || context.Response.StatusCode == 403)">
                <cache-remove-value key="api0000-demo-conference-bearerToken" />
            </when>
        </choose>
        -->

If more then one backends is used, reference to name values and backend in a policy filem following naming need to be used {{[name]-client-id}}, Ex: {{api0000-demo-conference-extra-client-secret}}.

Ex: getsessions-operationPolicy.xml with grant type Client Credentials

<policies>
    <inbound>
    <base />
        <cache-lookup-value key="@("api0000-demo-conference-sessions-bearerToken")" variable-name="bearerToken" />
        <choose>
            <when condition="@(!context.Variables.ContainsKey("bearerToken-sessions"))">
                <send-request ignore-error="true" timeout="20" response-variable-name="accessTokenResult" mode="new">
                    <set-url>{{api0000-demo-conference-sessions-tokenUrl}}</set-url>
                    <set-method>POST</set-method>
                    <set-header name="Accept" exists-action="override">
                        <value>*/*</value>
                    </set-header>
                    <set-header name="Accept-Encoding" exists-action="override">
                        <value>gzip, deflate, br</value>
                    </set-header>
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/x-www-form-urlencoded</value>
                    </set-header>
                    <set-body>@{
                                return "client_id={{api0000-demo-conference-sessions-client-id}}&client_secret={{api0000-demo-conference-sessions-client-secret}}&scope={{api0000-demo-conference-sessions-scope}}&grant_type=client_credentials";
                        }</set-body>
                </send-request>
                <set-variable name="accessToken" value="@(((IResponse)context.Variables["accessTokenResult"]).Body.As<JObject>())" />
                <set-variable name="bearerToken" value="@((string)((JObject)context.Variables["accessToken"])["access_token"])" />
                <set-variable name="tokenDurationSeconds" value="@((int)((JObject)context.Variables["accessToken"])["expires_in"])" />
                <cache-store-value key="api0000-demo-conference-sessions-bearerToken" value="@((string)context.Variables["bearerToken"])" duration="@((int)context.Variables["tokenDurationSeconds"])" />
            </when>
        </choose>
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["bearerToken"])</value>
        </set-header>
    <set-backend-service backend-id="api0000-demo-conference-sessions" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
        <choose>
            <when condition="@(context.Response.StatusCode == 401 || context.Response.StatusCode == 403)">
                <cache-remove-value key="api0000-demo-conference-sessions-bearerToken" />
            </when>
        </choose>    
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

IMPORTANT! The Azure API Management need to be given GET access to the secrets in the Key Vault if Key Vault is being used. This is done by giving the Azure API Managements systems assigned identity access to the KeyVault.

API Management Managed Identity

LKAB Internal DNS names

LKABs Azure environment will resolve LKAB internal DNS names, e.g. my-api-server.corp.lkab.com, to LKAB internal IP addresses, e.g. 10.x.x.x.
As long as the firewall openings are in place for the given API Management instance to access the given backend server IP address, your api can use LKAB internal hostnames as backend.
E.g. in your api's params-{env}.json file:

"backendEndpointUrl": {
      "value": "https://lkapiprod01.corp.lkab.com/ip21/v3-aad"
    },

Step 7 - Create pull request

Go to Azure DevOps, select your project and select Repos
1. If you don't self-host, you'll find the repo in the api-management-apis project
Select your API repository.
Select Pull requests
Select Create a pull request or New pull request
Make sure that the feature branch is pulled into the develop branch.
1. Add a Title
2. Add a Description
3. Add Work items to link, Connect the work item that was created in the beginning.
Optional: Click Set auto-complete
Wait for Review and solve any review comments. When all checks are green go-to point 8
Select the pull request, if auto-complete were selected go to point 10.
Select Complete on the pull request.
Go to the pipeline and verify that it turns green. If it's waiting for permission, see below.

You'll have to manually permit the pipeline to use the dev-azure environment to complete the Azure Development stage.

Click on the View button.
Click on Permit both in the Waiting for review and Permit access? dialogs.

It is also possible to permit the pipeline to use all environments beforehand.

Select Environments under Pipelines in the leftmost menu in DevOps
Click on the environment dev-azure and then select Security under the three-dot menu (More actions)
Click the plus sign on the Pipeline permissions panel and select the API deployment pipeline.
Repeat for the other environments: test-azure, stage-azure and prod-azure.

Step 8 - Move to Test and Stage

When the API is ready for test, this step SHOULD be performed.

Go to Azure DevOps, select your project and select Repos
Select your API repository.
Select Pull requests
Select Create a pull request or New pull request
Make sure that the develop is pulled into the release branch.
1. Add a Title
2. Add a Description
3. Add Work items to link, Connect the work item that was created in the beginning.
Optional: Click Set auto-complete
Wait for Review and solve a review comments. When all Checks is green go-to point 8
Select the pull request, if auto-complete were selected go to point 10.
Select Complete on the pull request.
Go to the pipeline and verify that it turns green. Maybe you will have to permit the usage of test-azure (and stage-azure) environment.
Optional: If the test is working fine and a decision is taken to move it forward to Stage, open the running pipeline and under Review select Approve and it will be deployed to Stage.

Step 9 - Move to Production

Go to Azure DevOps, select your project and select Repos
Select your API repository.
Select Pull requests
Select Create a pull request or New pull request
Make sure that the release is pulled into the master branch.
1. Add a Title
2. Add a Description
3. Add Work items to link, Connect the work item that was created in the beginning.
Optional: Click Set auto-complete
Wait for Review and solve a review comments. When all Checks is green go-to point 8
Select the pull request, if auto-complete were selected go to step 10.
Select Complete on the pull request.
APPROVAL Go to [pipeline] select the pipeline and approve production deployment.
Go to [pipeline] select the pipeline and verify that it turns green. Maybe you will have to permit the usage of prod-azure environment.