Develop and publish contract-first API to LKABs API Management platform
This guide shows how to develop and publish an API based on an OpenAPI Specification into LKABs API Management platform.
Release Cycle
Each API SHOULD
be stored in a GIT repository with a dedicated release pipeline. The release process is based on 3 main branches that each points to one or many API Management environments. All work should be done using a feature branch that then is merged into the developed branch and then pulled downstream to the next branch.
Branch | Will be deplyed to |
---|---|
develop | development environment apim-apimgmt-dev-weu-001-lkab |
release | test- and stage environments apim-apimgmt-test-weu-001-lkab and apim-apimgmt-stage-weu-001-lkab |
master | production environment apim-apimgmt-prod-weu-001-lkab |
When developing the API will pass 3 maturity steps.
- Experimental → develop
- Unstable → release
- Stable → master/main
When the feature branch is pushed to the remote repository a build is started where the code is checked using a linters and a build is performed.
Skipping environments
It is often the case that the backend api doesn't have the corresponding environments. For example the backend only have a test and production. Then you probably just want to handle those two environments in API Management. This can be done by changing the generated azure-pipeline.yaml
in your api repository.
SKIP_DEV_DEPLOYMENT: true|false
SKIP_TEST_DEPLOYMENT: true|false
SKIP_STAGE_DEPLOYMENT: true|false
SKIP_PROD_DEPLOYMENT: true|false
...
stages:
- template: ./src/pipeline-templates/.azure-pipelines-api-deployment.yml@templates-apim
parameters:
DEPLOYMENT_NAME: $(Build.Repository.Name)
MARKDOWN_LINT_ENABLED: ${{parameters.markdownlintEnabled}}
OAS_LINT_ENABLED: ${{parameters.oaslintEnabled}}
LOCATION: 'westeurope'
SKIP_DEV_DEPLOYMENT: false
SKIP_TEST_DEPLOYMENT: false
SKIP_STAGE_DEPLOYMENT: false
SKIP_PROD_DEPLOYMENT: false
Branch strategies when skipping environments
Some common use cases and suggestions
Only production backend
SKIP_DEV_DEPLOYMENT: true
SKIP_TEST_DEPLOYMENT: true
SKIP_STAGE_DEPLOYMENT: true
SKIP_PROD_DEPLOYMENT: false
Only work with pull requests against master/main
Only test and production backend
SKIP_DEV_DEPLOYMENT: false
SKIP_TEST_DEPLOYMENT: true
SKIP_STAGE_DEPLOYMENT: true
SKIP_PROD_DEPLOYMENT: false
Only work with pull requests against develop
during development and master/main
for production.
Linter tools:
The code is then brought in from the feature branch making a pull request against the develop branch. To complete the pull request all checks need to be in success state.
- The pull needs to have a work item.
- The pull request needs to be reviewed and approved by at least one other team member in the Azure DevOps project.
- If there are any comments those needs to be resolved.
- Build must have succeeded.
Suggested tools to install
Firewalls
LKAB API Management platform will need access to the API backend that is to be exposed and requires firewall openings.
- Azure APIs and other public APIs requires firewall openings in Azure Firewall.
- On-premise API requires firewall openings in the On-premise data center firewall. See the Firewall guide here. Request for these openings as early as possible.
Develop and publish API
Step 1 - Reserve an API Id and basepath
Each API SHOULD
have an unique API Id on the format api[0000-9999]-[api name]
, as well as an unique API basepath.
The API basepaths are globally unique within the API Management platform. To avoid collisions, the basepath assigned to an API published to the LKAB API Management platform, need to conform with the pre-defined set of basepaths in the Pre-defined basepaths document, and be unique within the API Catalog.
Assignement of API basepath is part of the onboarding process when a new API are about to be published to the API Management platform, agreed upon between API owner and the API Management team.
The agreed and assigned API basepath SHOULD
be documented along with the assigned API Id in the API Catalog before going further in the deployment process.
Step 2 - Create work item
To collect information about api:s and track the progress of the onboarding process, all api:s should have a Product Backlog Item (PBI) on the api-management-apis taskboard. If a PBI already exist, continue with Step 3.
To create a new PBI:
- Create a PBI via this PBI creation link
- Update the title
- Add work item 35358 New Apis as parent
- Fill in the form in the description (can be edited later too)
- Save
- Add tasks via 1-Click Child-Links action
Some tasks refers back to this document, helping you completing the tasks.
Step 3 - Create repository
Follow the instructions in this section.
Step 4 - API Information
Replace the OpenAPI Specification in the [src/openapi] folder with the OpenAPI Specification for your API and ensure the OpenAPI Specification to be compliant with Stoplight spectral-cli linter.
The file SHOULD
be named oas.yml
.
NOTE! There is currently a size limitation on the oas file due to the usage of the function loadYamlContent, see Issue: Increase MaxLiteralCharacterLimit for loadYamlContent. Maybe it's possible to add the oas content inline in the
bicep/api-definition.bicep
file.
Common OpenAPI updates
These are some common updates that are needed to make the OpenAPI Specification valid.
- Add tags, contact and change host name of url
- Replace summary name with description on the operations
- Change description for the response 200 to "Success"
- Add empty tags element to operations that are missing this
openapi: 3.0.1
info:
title: LKAB.Test.API
version: '1.0'
contact: # Add this section
name: <name of the technical contact>
email: <email of the technical contact>
description: LKAB Test API
servers:
- url: https://api-lkab.lkab.com/test/connectivity # Change host
tags: # add tags section and tags used on operations
- name: WeatherForecast
paths:
"/WeatherForecast":
get:
tags:
- WeatherForecast # If this tag exist, tag need to be added in tags section above
operationId: GetWeatherForecast
description: Get Weather Forecast # Summery to description if missing, or add a description
responses:
'200':
description: Success # Add a valid description
Tip: run
spectral lint **/openapi/*.* --ruleset ./.spectral.yml
neednpm i @stoplight/spectral-cli -g
in the root of the repo
Step 5 - Update configuration information
Update the...
- OpenAPI Specification in the repo folder
src/openapi
, this should have been done already in the previous step - pipeline id in the README.MD file located in the root folder of the repo. Replace
TODO:addPipelineId
with the pipeline id of the pipeline in Azure DevOps. Also, if you have a self-hosted repo (ie the repo is not located in the api-management-apis DevOps project), you need to update the URL path and replace the api-management-apis part with your project name. - OPTIONAL: Update
api-definition.bicep
located in the repo foldersrc/bicep/
file with policy if there is a need to have separate policy handling for operations. See section policies, below for more information about how to do this. Parameter files
located in the repo foldersrc/bicep/environments/[regions: westeurope]
Note! At the moment we are just supportingwesteurope
, See Parameter files for mor information- Optional: Run command
markdownlint **/*.md --config ./.markdownlint.yml
locally to check that markdown is following markdown standard. Note! This requires that the Markdownlint-cli has been installed. Fix markdown errors if any. - Commit the changes and push to remote repository.
git commit -am "Configure and publish api"
andgit push
Policies
Parameter files
The parameter files for each environment need to be updated.
Examples exist in the api-definition.dev.bicepparam
locates in teh folder src/bicep/environments/[regions: westeurope]
Allowed Origins (OPTIONAL)
If origins need to be allowed uncomment the allowed Origins parameter.
// Add allowed origins if CORS is needed, if set a cors policy fragments is added
param allowedOrigins = [
'foo.com' // Try to avoid *
]
Update backends
These are the backend security types that are currently supported out of the box.
Backend protected with:
- Managed Identity, this is the recommended backend security to use when possible (Can be used with almost all Azure Native services)
- Bearer token grant type
Client Credentials
- Bearer token grant type
Password
- Bearer token grant type
Refresh Token
- Behalf Of The Authenticated Identity, this is used when the JWT i checked in the API and then should be passed forward to the backend.
- API Key, this
SHOULD BE AVOIDED
if possible, ask for another backend security type. - Bearer token grant type
Client Credentials (Static token)
, thisSHOULD BE AVOIDED
if possible but sometimes we get a static bearer to use. This we should consider as bad as a Basic Authentication - Basic Authentication, this
SHOULD BE AVOIDED
if possible, ask for another backend security type.
Add backend that is needed by the API by uncomment the bicep fragment in the parameter file that match the correct backend security type.
NOTE! If more then one backend is use the name value MUST be set for the 2...n backends, the
repoName
part inapiPolicy.xml
file located in the repo foldersrc/bicep/policies
need to be changed to[repoName]-[backend-name]
. If only one backend is used it is recommended to not include the name parameter in the backend.
Ex. One backend:
param bearerTokenGrantTypeClientCredentialsBackends = [
{
endpointUrl: 'https://conferenceapi.azurewebsites.net'
clientId: '11111111-1111-1111-1111-111111111111'
#disable-next-line no-hardcoded-env-urls
clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'
// NOTE!
// If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other.
// This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
// then the tokenUrl should not be set
#disable-next-line no-hardcoded-env-urls
tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
// Optional
description: 'The api0000-demo-conference backend info'
scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
}
]
Ex. More then one: NOTE! The name in the second object
param bearerTokenGrantTypeClientCredentialsBackends = [
{
endpointUrl: 'https://conferenceapi.azurewebsites.net'
clientId: '11111111-1111-1111-1111-111111111111'
#disable-next-line no-hardcoded-env-urls
clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'
// NOTE!
// If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other.
// This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
// then the tokenUrl should not be set
#disable-next-line no-hardcoded-env-urls
tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
// Optional
description: 'The api0000-demo-conference backend info'
scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
}
{
endpointUrl: 'https://conferenceapi.azurewebsites.net'
clientId: '11111111-1111-1111-1111-111111111111'
#disable-next-line no-hardcoded-env-urls
clientSecretKvUri: 'https://kvapimdev001lkab.vault.azure.net/secrets/api0000-secret'
// NOTE!
// If the same token endpoint is used, the <set-url> value can be hardcoded in the apiPolicy.xml or any other.
// This is a prefered to have less name values. Example use <set-url>https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token</set-url>
// then the tokenUrl should not be set
#disable-next-line no-hardcoded-env-urls
tokenUrl: 'https://login.microsoftonline.com/lkabonline.onmicrosoft.com/oauth2/v2.0/token'
// Optional
description: 'The api0000-demo-conference backend info'
scope: 'https://test.api.gs-ic.com/external/api0000/.default' // Note if Scope is not used, the scope query string need to be remove from apiPolicy.xml bakend policy fragment
name: 'api0000-demo-conference-backend-sessions'
}
]
Make similar changes in the parameter files for for test, stage and prod, where the information is related to each environment. Example on how to reference backends is found in the apiPolicy.xml
file located in the repo folder src/bicep/policies
.
Uncomment the backend policy fragments with the correct "grant type" in the
apiPolicy.xml
, uncommented policy fragment that is not needed can be removed.
Ex: Client Credentials, uncomment this two sections between <!--
and -->
<!-- #### GRANT TYPE Client Credentials: bearerTokenGrantTypeClientCredentialsBackends, Example
<cache-lookup-value key="@("api0000-demo-conference-bearerToken")" variable-name="bearerToken" />
<choose>
<when condition="@(!context.Variables.ContainsKey("bearerToken"))">
<send-request ignore-error="true" timeout="20" response-variable-name="accessTokenResult" mode="new">
<set-url>{{api0000-demo-conference-tokenUrl}}</set-url>
<set-method>POST</set-method>
<set-header name="Accept" exists-action="override">
<value>*/*</value>
</set-header>
<set-header name="Accept-Encoding" exists-action="override">
<value>gzip, deflate, br</value>
</set-header>
<set-header name="Content-Type" exists-action="override">
<value>application/x-www-form-urlencoded</value>
</set-header>
<set-body>@{
return "client_id={{api0000-demo-conference-client-id}}&client_secret={{api0000-demo-conference-client-secret}}&scope={{api0000-demo-conference-scope}}&grant_type=client_credentials";
}</set-body>
</send-request>
<set-variable name="accessToken" value="@(((IResponse)context.Variables["accessTokenResult"]).Body.As<JObject>())" />
<set-variable name="bearerToken" value="@((string)((JObject)context.Variables["accessToken"])["access_token"])" />
<set-variable name="tokenDurationSeconds" value="@((int)((JObject)context.Variables["accessToken"])["expires_in"])" />
<cache-store-value key="api0000-demo-conference-bearerToken" value="@((string)context.Variables["bearerToken"])" duration="@((int)context.Variables["tokenDurationSeconds"])" />
</when>
</choose>
<set-header name="Authorization" exists-action="override">
<value>@("Bearer " + (string)context.Variables["bearerToken"])</value>
</set-header>
-->
and
<!--
#### GRANT TYPE:
bearerTokenGrantTypeClientCredentialsBackends, Example
bearerTokenGrantTypePasswordBackends, Example
bearerTokenGrantTypeRefreshTokenBackends, Example
<choose>
<when condition="@(context.Response.StatusCode == 401 || context.Response.StatusCode == 403)">
<cache-remove-value key="api0000-demo-conference-bearerToken" />
</when>
</choose>
-->
If more then one backends is used, reference to name values and backend in a policy filem following naming need to be used {{[name]-client-id}}, Ex: {{api0000-demo-conference-extra
-client-secret}}.
Ex: getsessions-operationPolicy.xml with grant type Client Credentials
<policies>
<inbound>
<base />
<cache-lookup-value key="@("api0000-demo-conference-sessions-bearerToken")" variable-name="bearerToken" />
<choose>
<when condition="@(!context.Variables.ContainsKey("bearerToken-sessions"))">
<send-request ignore-error="true" timeout="20" response-variable-name="accessTokenResult" mode="new">
<set-url>{{api0000-demo-conference-sessions-tokenUrl}}</set-url>
<set-method>POST</set-method>
<set-header name="Accept" exists-action="override">
<value>*/*</value>
</set-header>
<set-header name="Accept-Encoding" exists-action="override">
<value>gzip, deflate, br</value>
</set-header>
<set-header name="Content-Type" exists-action="override">
<value>application/x-www-form-urlencoded</value>
</set-header>
<set-body>@{
return "client_id={{api0000-demo-conference-sessions-client-id}}&client_secret={{api0000-demo-conference-sessions-client-secret}}&scope={{api0000-demo-conference-sessions-scope}}&grant_type=client_credentials";
}</set-body>
</send-request>
<set-variable name="accessToken" value="@(((IResponse)context.Variables["accessTokenResult"]).Body.As<JObject>())" />
<set-variable name="bearerToken" value="@((string)((JObject)context.Variables["accessToken"])["access_token"])" />
<set-variable name="tokenDurationSeconds" value="@((int)((JObject)context.Variables["accessToken"])["expires_in"])" />
<cache-store-value key="api0000-demo-conference-sessions-bearerToken" value="@((string)context.Variables["bearerToken"])" duration="@((int)context.Variables["tokenDurationSeconds"])" />
</when>
</choose>
<set-header name="Authorization" exists-action="override">
<value>@("Bearer " + (string)context.Variables["bearerToken"])</value>
</set-header>
<set-backend-service backend-id="api0000-demo-conference-sessions" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
<choose>
<when condition="@(context.Response.StatusCode == 401 || context.Response.StatusCode == 403)">
<cache-remove-value key="api0000-demo-conference-sessions-bearerToken" />
</when>
</choose>
</outbound>
<on-error>
<base />
</on-error>
</policies>
IMPORTANT! The Azure API Management need to be given GET access to the secrets in the Key Vault if Key Vault is being used. This is done by giving the Azure API Managements systems assigned identity access to the KeyVault.
LKAB Internal DNS names
LKABs Azure environment will resolve LKAB internal DNS names, e.g. my-api-server.corp.lkab.com
, to LKAB internal IP addresses, e.g. 10.x.x.x
.
As long as the firewall openings are in place for the given API Management instance to access the given backend server IP address, your api can use LKAB internal hostnames as backend.
E.g. in your api's params-{env}.json file:
"backendEndpointUrl": {
"value": "https://lkapiprod01.corp.lkab.com/ip21/v3-aad"
},
Step 7 - Create pull request
- Go to Azure DevOps, select your project and select Repos
- If you don't self-host, you'll find the repo in the api-management-apis project
-
Select your API repository.
-
Select
Pull requests
- Select
Create a pull request
orNew pull request
- Make sure that the
feature branch
is pulled into thedevelop
branch.- Add a
Title
- Add a
Description
- Add
Work items to link
, Connect the work item that was created in the beginning.
- Add a
- Optional: Click
Set auto-complete
- Wait for Review and solve any review comments. When all checks are green go-to
point 8
- Select the pull request, if
auto-complete
were selected go topoint 10
. - Select
Complete
on the pull request. - Go to the pipeline and verify that it turns green. If it's waiting for permission, see below.
You'll have to manually permit the pipeline to use the dev-azure
environment to complete the Azure Development
stage.
-
Click on the
View
button. -
Click on
Permit
both in the Waiting for review and Permit access? dialogs.
It is also possible to permit the pipeline to use all environments beforehand.
- Select Environments under Pipelines in the leftmost menu in DevOps
- Click on the environment dev-azure and then select Security under the three-dot menu (More actions)
- Click the plus sign on the Pipeline permissions panel and select the API deployment pipeline.
- Repeat for the other environments: test-azure, stage-azure and prod-azure.
Step 8 - Move to Test and Stage
When the API is ready for test, this step SHOULD
be performed.
- Go to Azure DevOps, select your project and select Repos
- Select your API repository.
- Select
Pull requests
- Select
Create a pull request
orNew pull request
- Make sure that the
develop
is pulled into therelease
branch.- Add a
Title
- Add a
Description
- Add
Work items to link
, Connect the work item that was created in the beginning.
- Add a
- Optional: Click
Set auto-complete
- Wait for Review and solve a review comments. When all Checks is green go-to
point 8
- Select the pull request, if
auto-complete
were selected go topoint 10
. - Select
Complete
on the pull request. - Go to the pipeline and verify that it turns green. Maybe you will have to permit the usage of test-azure (and stage-azure) environment.
- Optional: If the test is working fine and a decision is taken to move it forward to
Stage
, open the running pipeline and under Review select Approve and it will be deployed toStage
.
Step 9 - Move to Production
- Go to Azure DevOps, select your project and select Repos
- Select your API repository.
- Select
Pull requests
- Select
Create a pull request
orNew pull request
- Make sure that the
release
is pulled into themaster
branch.- Add a
Title
- Add a
Description
- Add
Work items to link
, Connect the work item that was created in the beginning.
- Add a
- Optional: Click
Set auto-complete
- Wait for Review and solve a review comments. When all Checks is green go-to
point 8
- Select the pull request, if
auto-complete
were selected go tostep 10
. - Select
Complete
on the pull request. - APPROVAL Go to [pipeline] select the pipeline and approve production deployment.
- Go to [pipeline] select the pipeline and verify that it turns green. Maybe you will have to permit the usage of prod-azure environment.