Sanjeewa Malalgoda's Blog: March 2014

Fixing issue in WSO2 API Manager due to matching resource found or API authentication failure for a API call with valid access token

No matching resource found error or authentication failure can happen due to few reasons. Here we will discuss about errors can happen due to resource define

Here in this article we will see how resource mapping work in WSO2 API Manager. When you create API with resource we will store them in API Manager database and synapse configuration. When some request comes to gateway it will first look for matching resource and then dispatch to inside that. For this scenario resource is as follows.

/resource1/*

In this configuration * means you can have any string(in request url) after that point. If we take your first resource sample then matching request would be something like this.

http://gatewayhostname:8280/t/test.com/apicontext/resource1/

Above request is the minimum matching request. In addition to that following requests also will map to this resource.

http://gatewayhostname:8280/t/test.com/apicontext/resource1/data?name=value

http://gatewayhostname:8280/t/test.com/apicontext/resource1/data?name=value&id=97

And following requests will not map properly to this resource. The reason for this is we specifically expecting /resource1/ in the request(* means you can have any string after that point).

http://gatewayhostname:8280/t/test.com/apicontext/resource1?name=value&id=97

From the web service call you will get following error response.

<am:fault xmlns:am="403Status'>http://wso2.org/apimanager"><am:code>403</am:code><am:type>Status report</am:type><am:message>Runtime Error</am:message><am:description>No matching resource found in the API for the given request</am:description></am:fault>

If you sent request to t/test.com/apicontext/1.0/resource1?id=97&Token=cd it will not work. Because unfortunately there is no matching resource for that. Because as i explained earlier you resource definition is having /resource1/*. Then request will not map to any resource and you will get no matching resource found error and auth failure(because trying to authenticate against none existing resource).

Solution for this issue would be something like this.

In API Manager we do support both uri-template and url mapping support. If you create API from API Publisher user interface then it will create url-mapping based definition. From API Manager 1.7.0 on wards we will support both options from UI level. Normally when we need to do some kind of complex pattern matching we use uri-template. So here we will update synapse configuration to use uri-template instead of url-mapping. For this edit wso2admin-AT-test.com--apicontext_v1.0.xml file as follows.

Replace <resource methods="GET" url-mapping="/resource1/*"> with <resource methods="GET" uri-template="/resource1?*">

Hope this will help you to to understand how resource mapping work. You will find more information from this link[1]

[1]http://charithaka.blogspot.com/2014/03/common-mistakes-to-avoid-in-wso2-api.html

How to configure WSO2 API Manager to access by multiple devices(from single user and token) at the same time

This would be very useful when we setup production kind of deployments and use it by many users. According to current architecture if logged out from one device and revoke key then all other call made with that token will get authentication failures. In that case application should be smart enough to detect authentication failure and request for new token. Once user log into application, that user might want to provide user name and password. So we can use that information and consumer/ secret keys to retrieve new token once detect authentication failure. In our honest opinion we should handle this from client application side. If we allowed users to have multiple tokens at the same time. And that will cause to security related issues and finally users will end up with thousands of tokens that user cannot maintain. Also this might be problem when it comes to usage metering and statistics.

So recommended solution for this issue is having one active user token at a given time. And make client application aware about error responses send by API Manager gateway. Also you should consider refresh token approach for this application. When you request user token you will get refresh token along with the token response so you can use that for refresh access token.

How this should work

Lets assume same user logged in form desktop and tablet. Client should provide user name and password both when they log into desktop and tablet apps. At that time we can generate token request with username, password and consumer key, secret pair. So we can keep this request in memory until user close or logout from application(we do not persist this data to anywhere then there is no security issue)

then when they logout from the desktop or the application on the desktop decides to refresh the OAuth Token first, then the user will be prompted for their username and password on the tablet since the tablet has a revoked or inactivated OAuth Token. But here we should not prompt username password as client is already provided them and we have token request in memory. Once we detect auth failure from tablet app it will immediately send token generation request and get new token. User will not aware about what happen underline.

How to move WSO2 API Management platform from one to another deployment( API Manager-1.6.0)

It seems this is very valid requirement coming from API manager users. At this moment we do not have deployable API artifacts. When it comes to API management related artifacts its bit different from .war or .car archives. Because when we migrate API platform from one to another we need to migrate APIs, Applications, tokens, usage data and etc. And its not simple as we deploy war in another deployment. For the moment you can use following work around.

01. Create setup for QA environment.
02. Move API Manager database to production system after QA process completes.
03. Move User management related database or LDAP to production system after QA process completes,
04. Move registry database to production system after QA process completes.
05. Move repository/deployment/server folder to production system after QA process completes (run time artifacts migration) . If you have tenants in your system you might need to move tenant specific data as well. For that move /repository/tenants (If you have tenants in your setup).
06. In addition to that if you are using BAM then move usage data as well(but recommend to use fresh usage data store for production as you don't need to mix up production and QA statistics).

Is seems bit complicate at the start but actually its very easy and reliable way of migrating between environments(actually it need database dump and restore + copy run time artifacts to new deployment). You can easily puppetize this or can have your own deployment tool.

How to fix issue in WSO2 API Manager due to missing API resource

When you visit subscription page in API manager store page you might see following error.

org.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: Resource does not exist at path /_system/governance/apimgt/applicationdata/provider/admin/api-name/1.0.0/api
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.get(EmbeddedRegistry.java:532)

The reason for this issue is API resource is not there in provided path(/_system/governance/apimgt/applicationdata/provider/admin/api-name/1.0.1/api ). Here, subscriptions are run time data that stored in api manager db and API are meta data stored in registry. This error log says application is having API which is not there in registry.

This can happen due to multiple reasons.

01. Deleting registry API resource without deleting it from publisher UI(then it will delete registry data and entries in API manager tables).

02. change governance space after create subscriptions.

For this issue you can do followings.

If governance space changed you can mount to old space again. Then missed API will appear in given path.

You can create missed api-name/1.0.1 again in the system. For this delete API from api manager tables and create API from publisher UI.

You can delete subscriptions associated with missed API(api-name/1.0.1). For this you need to delete all associated entries from API manager tables carefully(or you can drop API manager database and create new database if you are not in production system ).

Actually we don't need to delete entire database. Please run following db queries against you API manager database. Its always recommend to run these queries on test environment and then apply to production servers.

Here first we need to get API_ID associated with problematic API. To get that please run following command

SELECT API_ID FROM AM_API where API_NAME = 'api-name' and API_VERSION = '1.0.1'

Then we need to delete all subscriptions associated with that API. Let say we got 3 as the result of above query. Run following query to delete all associated subscriptions.

DELETE FROM AM_SUBSCRIPTION where API_ID = '3'

How to revoke access tokens programmatically In API Manager 1.4.0

Here in this article we will see how we can revoke tokens manually using curl command or any other client. We have rest endpoint to revoke access token. For this you need to provide required information with revoke request. See following sample curl commands. Hope you will be able to implement your client based on those requests. You need to pass token to be revoked, consumer key and authorized user as parameters.

Login to Publisher:
curl -X POST -c cookieshttp://localhost:9763/publisher/site/blocks/user/login/ajax/login.jag -d 'action=login&username=admin&password=admin'

Revoke Tokens:
curl –X POST -b cookieshttp://localhost:9763/publisher/site/blocks/tokens/ajax/revokeToken.jag -d "action=revokeAccessToken&accessToken=hLmK_5TvX6f2NiSXkZ3h_l2NpnIa

&consumerKey=DTdSX2GhCCmK69jttYbTigyIxKka&authUser=admin"

How to customize WSO2 API Manager swagger UI to invoke APIs without auth token

In this post we will see how we can skip providing auth keys when we invoke APIs using swagger user interface(In WSO2 API Manager 1.6.0). One possible approach is marking resource auth type as none. if you marked resource level authentication as none when you create API then you can invoke it through swagger UI without having keys.

Lets look in detail other option. For this we might need to do small customization to swagger users interface and some additional steps.
01. Add all APIs that need to be invoke through swagger to single application and generate token with some large life time.
02. Then hard code that token to swagger user interface jaggery js file as instructed below.
     Open and edit file wso2am-1.6.0/repository/deployment/server/jaggeryapps/store/site/themes/fancy/templates/api/swagger/template.jag
     Add following generated access token as follows next to supported methods section.
     headers: { 'Authorization': 'Bearer 7c7a62dd139b819776ea06f845cd48f'},
     Then content will be like this
     window.swaggerUi = new SwaggerUi({
                discoveryUrl:"<%=discoveryUrl%>",
                dom_id:"swagger-ui-container",
                apiKeyName: "authorization",
                supportHeaderParams: true,
                supportedSubmitMethods: ['get', 'post', 'put', 'delete', 'options'],
                headers: { 'Authorization': 'Bearer 7c7a62dd139b819776ea06f845cd48f'},
                onComplete: function(swaggerApi, swaggerUi){

03. Then you will be able to invoke APIs without tokens. But in swagger UI we have mandatory field to enter auth token. We can put some random value and invoke APIs(actually hard corded token will be used).
     But if you wish to remove it follow below instructions.
         Go to publisher UI and select the API you interested. Then go to docs section and edit Swagger API Definition as follows.
         You will see for each resource section parameter named Authorization ("name": "Authorization")
         By default that field is marked as required filed. You have to remove it. Then content would be like this
                       {
                           "name": "Authorization",
                            "description": "Access Token",
                            "paramType": "header",
                            "allowMultiple": false,
                            "dataType": "String"
                        },

Now you will be able to invoke APIs without providing keys using swagger UI.

WSO2 API Manager workflow extension support

This was a major requirement since early stages of WSO2 API Manager and we decided to add it to API manager 1.6.0 release. With this new feature users will be able to plug workflow extensions in to APIManager major operations. In this release(1.6.0) we introduced workflow extension support for three major operations of API Manager. In order to try this you need API Manager 1.6.0 or above version and WSO2 Business Process server 3.0.0 or above. We don't expect high level business process workflow and human task design knowledge to try this as we embedded all necessary workflows and human tasks into product distribution. We have documented about this feature in API manager document page(link to document 1.6.0). Here in this article we will discuss about API Manager workflow extensions design, main components, configurations and usages of it. And finally we will see how we can customize this workflow process according to our requirement.

01. Introduction.

This was a major requirement since early stages of WSO2 API Manager and we decided to add it to API manager 1.6.0 release. With this new feature users will be able to plug workflow extensions in to APIManager major functionalities. In this release(1.6.0) we introduced workflow extension support for three major operations of API Manager. In order to try this you need API Manager 1.6.0 or above version and WSO2 Business Process server 3.0.0 or above. We don't expect high level business process workflow and human task design knowledge to try this as we embedded all necessary workflows and human tasks into product distribution. We have documented about this feature in API manager document page(link to document 1.6.0). Here in this article we will discuss about API Manager workflow extensions design, main components, configurations and usages of it. And finally we will see how we can customize this workflow process according to our requirement.

If we enabled Workflow Extension feature then we will not allow to complete user sign-up, application creation and subscription processes. Instead of that we will be keeping in an intermediate state and let some administrator to approve the action. With this new implementation we will be able to get certain actions if users performed any of above mentioned actions. With this system administrators can have some kind of control over user actions. It works like this, if user performed some action then approval task will handover to business process. If we take user creation as an example when new user is created on the system he will not be able to login to the system immediately, instead he will see message saying "User account creation wait for administrator approve". Once administrator approves then only user will be able to login to the system and use API store functionalities. Actually what happened here is, when new user created in the system we will call external business process with necessary information and waiting for call back from it. We have added following additional steps before complete user creation( same applies application creation and subscription).

01. Call BPS workflow web service from API manager once configured event happened.

01. Accept workflow request coming from API manager to BPS.

02. Create human tasks instance in BPS.

03. User interaction to approve workflow. User can log in to "Workflow-Admin" jaggery app which is default ship with API manager 1.6.0 and approve process.The url for "workflow-admin" jaggery application is "http[s]://ip:port/workflow-admin"

04. Call back to API manager after workflow execution.

05. Complete action from API Manager side.

02. Workflow Extension plug points.

Next we will discuss about 3 plug points and importance of having them. These are 3 main plug points

User Signup: Whenever new user registered from self signin page this workflow will trigger (only if you enabled user self sign in feature by editing api-manager.xml). This would be very useful as administrators can review all user registrations before they use system. When users try to sign up to the API Manager Store, their request for user accounts will go into an intermediary state where in which it will remain until it get approve by administrative. To enable User Sign Up workflow you will need to configure the UserSign up executor by editing UserSignUpSimpleWorkflowExecutor section of <AM_HOME>/repository/conf/api-manager.xml file.

API Subscription: Whenever new subscription created this workflow will get executed. You can control how users can add APIs to applications created by them. Administrative users will get authority to control of API subscriptions. Once the workflow is engaged in the API subscription scenario, user's request for a new subscription will go into a temporary 'On Hold' state where in which it will remain until approved or rejected by the Workflow Administrator. To enable API Subscription workflow you will need to configure theSubscriptionCreation executor by editing "SubscriptionCreationSimpleWorkflowExecutor" section of <AM_HOME>/repository/conf/api-manager.xml file.

Application Creation: Once user created application and subscribe to APIs they can invoke APIs as they need. So each application created by users should be reviewed and controlled as they are main entry point to access secured APIs. When users try to create an Application in the API Manager Store, their request will go into an intermediary state where in which it will remain till authorized by the workflow authorization administrator. To enable the Application Creation workflow you will need to configure the ApplicationCreation executor by editing "ApplicationCreationSimpleWorkflowExecutor" section of <AM_HOME>/repository/conf/api-manager.xml file.

03. Components of Workflow Extension.

Next we will discuss about main components of Workflow Extension implementation. These are the three main components of workflow executor. These components are connected to each other through web services APIs.

01. workflow executor

Workflow executor is kind of a plug point to each process followed by API manager. As an example we can discuss about API creation, Application creation, user sign up and subscription creation. Each workflow executor in the WSO2 API Manager should be an implementation of org.wso2.carbon.apimgt.impl.workflow.WorkflowExecutor abstract class. Workflow executor class is having execute method and it will create payload according to predefined format. Then it will create web service client with the information we configured in api-manager.xml file (with workflow configuration Properties) and call business process service deployed in WSO2 BPS. Next we will see what business process is doing.

02. Business process

A workflow consists of a sequence of connected steps where each step follows. In this case we will create a business process in WSO2 Business Process Server and expose it as a secured web service to outside. So once workflow executor triggered we will find appropriate workflow for performed action and do a web service call with all provided information. As mentioned earlier external workflow would typically be a secured web service. Therefore the calling client (the API Manager in this case) requires to pass in necessary credentials to invoke the service. The configuration related to executing an external workflow would be as shown in configuration section of this article. This configuration can be found in the <AM_HOME>/repository/conf/api-manager.xml file.

[Business process image]

As you can see in this image (design view of workflow from developer studio) we have business process and human task for each workflow. Once this workflow received event from API manager it takes input parameters and create human task instances by calling associated human task web service. Once business process created human task, it will wait till some user approve it manually(human interaction). To approve pending human tasks we will call human tasks API from API manager. Once human task is completed business process will take output from completed human task and call back the API manager. As you can see in the diagram last invoke operation is there to callback API manager. Once callback receiver (at API Manager) received approval it will change the state of application to 'ACTIVE' ( if we considered application approval process. Otherwise it will change user's state or subscription state).

03. Human task

A human task is, quite simply, a unit of work done by a human. Quite often, this task involves the interaction with other services, and thus becomes a task within a larger business goal. If we take user sign up process as example, human task would be evaluate user details and approve user by administrator. Usually this action is performed by system administrator after reviewing information provided by user. For deployment we have used human task deployed in WSO2 BPS and API manager use BPS human task API to do approve pending tasks. Administrator may log into API manager jaggery application and approve pending human tasks.

API manager comes with default java workflow executors and a web service executor to trigger workflow processes for above three functions. These executors can be defined in api-manager.xml file and you can have your own custom implementation for that.If you wish to add your own workflow executor on each of these three processes, you can write your own custom executor and integrate it. Later in this article we will see how we can do it.

<DONE>

Configure workflow execution and use it

In this section we will see how we can configure API Manager with WSO2 Business process server with workflow execution. In order to run this we will assume users running API manager with port offset 0 and Business Process server with port offset 02.

Business Process Server Configurations

Set port offset as 2 in carbon.xml file of Business Process Server 3.0.0. Edit following element of carbon.xml file.

Copy epr folder in to repository/conf folder of Business Process Server.

Then copy human tasks (archived) files to repository/deployment/server/humantasks folder of Business Process Server.

Then copy business process (archived) files to repository/deployment/server/bpel folder of Business Process Server.

Then start server.

If you wish to change port offset to some other value or set hostname to a different value than localhost [when running BPS and AM in different machines] , then you will need to edit wsa:Address in all .epr files inside repository/conf/epr folder.

In addition to that we need to edit human tasks wsdl files as well. We need this because once human task finished it will call back to workflow. So it should aware about exact service location of workflow.

API Manager Configurations

Edit API manager configuration file to enable web service based workflow execution. For this we need to edit api-manager.xml located inside repository/conf of API Manager product.

All work flow related configurations are located inside <WorkFlowExtensions> section. Edit WorkFlowExtensions as follows. Please note that all workflow process services

are running on port 9765 of Business Process Server(as it is running with port offset2).

<Property name="serviceEndpoint">http://localhost:9765/services/ApplicationApprovalWorkFlowProcess</Property>

<Property name="username">admin</Property>

<Property name="password">admin</Property>

<Property name="callbackURL">https://localhost:8243/services/WorkflowCallbackService</Property>

</ApplicationCreation>

<Property name="serviceEndpoint">http://localhost:9765/services/SubscriptionApprovalWorkFlowProcess</Property>

<Property name="username">admin</Property>

<Property name="password">admin</Property>

<Property name="callbackURL">https://localhost:8243/services/WorkflowCallbackService</Property>

</SubscriptionCreation>

<Property name="serviceEndpoint">http://localhost:9765/services/UserSignupProcess</Property>

<Property name="username">admin</Property>

<Property name="password">admin</Property>

<Property name="callbackURL">https://localhost:8243/services/WorkflowCallbackService</Property>

</UserSignUp>

</WorkFlowExtensions>

To test this we will start with user creation process. Please go to API Manager store ui(http://127.0.0.1:9763/store). Then click on self signup button available in right top corner. Now you will be able to self sign up your self for API store. Once you provided all information required you will see following pop up window saying "User account waiting administrator approval"