Monday, August 15, 2016

WSO2 API Manager based solutions frequently asked questions and answers for them - 03

Can API Manager audit a source request IP address, and the user who made the request?
Yes, information on the request IP and user is captured and can be logged to a report.

Automation support for API setup and for configuring the API gateway?
Supported, setup can be automated through tools like Puppet. Puppet for common deployment patterns are also available for download.

Capability to run reports on API usage, call volume, latency, etc? 
Supported, API usage, call volume and latency can be reported. However information on caching is not reported.

Logging support and capability to integrate with 3rd party logging module?
By default our products use log4j for logging and if need we can plug custom log appenders. It is possible to push logs to an external system as well.

Billing and payment support & monitoring tools for API usage?
Billing and payment integration support is not available OOTB. But extension points are available to integrate with an external billing and payment systems. Currently WSO2 API Cloud (SaaS offering of WSO2 API Manager) is integrated with payment system successfully. So users can implement required extensions and get the job done.

Capability of message processing cycle control via pre/post processing?
Supported, it is possible to do some pre/post processing of messages based on what is available OOTB, however some pre/post processing would require custom capabilities to be written.

Does it support adapters or connectors to 3rd party systems such as Salesforce etc.
Supported by WSO2 ESB, WSO2 Integration Platform provides connectors to over 130 3rd party systems including Salesforce.com. The entire list of connectors can be accessed and downloaded from the following site.
https://store.wso2.com/store/assets/esbconnector"

Capability of monitoring and enforcing policy (i.e. message intercept).
Supported, it is possible to validate the message against a XSD to ensure that its compliant with a schema definition, it is also possible to audit and log messages and manage overall security by enforcing WS-Security on messages.

Multiple technology database support?
Database connectivity is provided via a JDBC adapter and multiple JDBC adapters can be used at the same-time. It is possible to change from one database technology to another as long as a JDBC adapter is available.

WSO2 API Manager based solutions frequently asked questions and answers for them - 02

Capability to create, manage and deploy both sandbox environments and production environments?
It is possible to manage a Sandbox and a Production environment simultaneously. Each of these environments can have its own API Gateway.

Can deploy application(s)/project(s) from one environment to another environment?
Applications and subscriptions cannot be migrate from one environment to another directly. But a RESTful API is available to get a list of applications and recreate them in another environment. However APIs can be imported/exported from one environment to another OOTB.

Capability to apply throttling to APIs and route the calls for different API endpoints?
Supported, Throttling can be applied to APIs based on a simple throttling rule such as number of requests allowed per minute or based on a complex rule which can consider multiple parameters such as payload size and requests per minute when throttling API calls.
API Gateway can apply throttling policies for different APIs and route the API calls for the relevant back end.

Supports  various versioning including URL, HTTP header, and Query parameter(s)?
API Manager supports URL based versioning strategy. If need we can implement our own

Capability to support API life cycle management including 'create', 'publish', 'block', and 'retire' activities?
API life cycle can be managed by the API Manager. By default it supports Created, Published, Depreciated, Blocked and Retired stages.

Can it manage API traffic by environments (i.e. sandbox, production etc.) and by Gateway?
Supported,Multiple API Gateways can be setup for Sandbox and Production environments to handle traffic of each environment separately. https://docs.wso2.com/display/AM200/Maintaining+Separate+Production+and+Sandbox+Gateways

Does it have throttling limit support?
Supported, Throttling enables users to create more complex policies by mixing and matching different attributes available in the message. Moreover, it supports throttling scenarios based on almost all header details. WSO2 API Manager 2.0 offers more flexibility when it comes to defining rules. In addition, the blocking feature will be very useful as well to protect servers from common attacks and abuse by users.

Provides rate limiting support?
Supported, rate limit support is available in the API Manager.

Capability to horizontally scale traffic in a clustered environment
Supported, a API Manager instance can handle over 3500 transactions per second when its fully optimized.

Support local caching for API responses (i.e. in non-clustered environment or when clustering not activated)?
Supported, it is possible to enable or disable API response caching for each API exposed.

Support distributed caching for API responses amongst nodes within a cluster?
Supported, caching is distributed to all Gateway nodes in the cluster.

Capability of auto-scaling via adding new nodes based on load ( i.e. auto spawning new instances and add to cluster)?
Autoscaling should be supported at underlying infrastructure level. The cluster allows any new node to join or leave the cluster whenever required.

Supports conversion from SOAP to REST?
Supported, SOAP to REST conversion is supported

Supports conversion from XML to JSON and JSON to XML within request and response payloads?
Supported, it is possible to convert the request and payload from XML to JSON and vice versa.
https://docs.wso2.com/display/AM200/Convert+a+JSON+Message+to+SOAP+and+SOAP+to+JSON

Supports redirecting API calls via the rewriting of URLs?
URL rewriting is supported with this it is possible to change final API destination dynamically based on a predefined condition and route requests. It is also possible to define parameterized URLs which can resolve a value in runtime according to an environment variable.

Ability to parse inbound URL for params including query parameters?
Supported, Query parameter, path parameter reading and modifications can be done before a request is sent to the backend.

Visual development, rapid mapping of activities and data?
Visual development and visual data mapper is available to develop the mediation sequences required. The visual development would be done via WSO2 Developer Studio which an Eclipse based IDE.

Custom activity - define custom code with input/output interface?
Supported, Custom code can be written as Java or Java scripts.

WSO2 API Manager based solutions frequently asked questions and answers for them - 01

OAuth support for REST API and WS-security token for SOAP/XML?
OAuth 2.0 support for REST API is available in WSO2 product platform. WS security, basic auth, xacml and other common authentication authorization implementations available in WSO2 solution and users can use them.

Support HMAC header signature or OAuth for REST API and WS-security message signature for SOAP service?
HMAC header signature validation and verification need to implement as separate handler and engage to API. That is not available as a OOTB solution. Both requirements can be fulfilled by writing a custom extension.

Support HTTPS for REST and ws-security message encryption for SOAP?
WSO2 supports WS Security and WS-Policy specifications. These specifications define a behavioral model for web services. A requirement for one proxy service may not be valid for another. Therefore, WSO2 provides the option to define service specific requirements. Security functionality is provided by the Security Management feature which is bundled by default in the Service Management feature of the WSO2 feature repository. The Security Management feature makes it easy to secure the proxy services of the ESB by providing twenty pre-defined, commonly-used security scenarios. These security policies are disabled by default.
https://docs.wso2.com/display/ESB481/Securing+Proxy+Services

Can safely publish APIs to external mobile applications?
Supported, APIs can be exposed to external mobile applications for consumption. Similarly a separate gateway can expose APIs for internal consumption as well.

Is it possible to integrate with external log systems or analyzing frameworks?
Users need to write custom log agent or use syslog agent to integrate with external tools. We have done similar integrations previously and use syslog agent for that.

Is it supports the 'API First' design with capabilities to publish API interfaces and/or import Swagger 2.0 definition(s)?
API first design capability is support with Swagger 2.0. It is possible to upload or refer a Swagger document and create the API based on this Swagger document.

Capability to support established open API documentation frameworks?
Supported, Swagger based documentation is supported. It is possible to create APIs based on a Swagger document. It is also possible to interactively test APIs through Swagger.

Can we deploy a prototyped API quickly without actual back end system?
"It is possible to deploy a new API or a new version of an existing API as a prototype.  It gives subscribers an early implementation of the API that they can try out without a subscription or monetization. Similarly in case if the actual back end service implementation is not available it is possible to create mock responses that can be sent as a response for API requests.
https://docs.wso2.com/display/AM200/Deploy+and+Test+as+a+Prototype"

Capability to browse and search APIs by provider, tag, name?
Supported, it is possible to search APIs and associated documentation as well as tag and filter APIs available in the portal.

Provides a developer portal with dashboard, subscription to API,  provisioning API keys capabilities & tokens?
API store (developer portal) provides a dashboard from which developers can  search, subscribe, generate API keys, rate and review APIs and view usage statistics from the API Store.

How API Manager supports community engagement with capabilities?
API Store (developer portal) allows users to self sign-up which can even be linked to an approval process where an authorized user's needs to approve the sign up before a user is allowed to access the portal. Users are able to view usage statistics, contribute towards the user forum, comment/review and rate APIs.

Capability to publish APIs to internal users only, external partners only and all users?
Supported, API visibility and subscription availability can be restricted based on user roles and tenants (if the API Manager is deployed in the multi-tenant mode).

Capability to provision API keys on demand?
Supported, API keys can be provisioned by users on-demand via the API Store or through the provided RESTful API.

Supports publishing and deploying APIs to multiple API Gateways?
Supported, it is possible to publish an API to a selected Gateway or to multiple Gateways. https://docs.wso2.com/display/AM200/Publish+through+Multiple+API+Gateways

Friday, August 12, 2016

How to manage multiple API Manager instances for different environment and use single external key Management server - WSO2 API Manager

Here i'm explaining solution for manage multiple API Manager environments and external key management server to build central API repository. Gateways in each environment can talk to different key managers as its just pointing key validation URLs.  In store we do not have concept of showing multiple consumer key/secret access tokens per environment. 
But like we discussed early we are using external key Manager as underlying key Manager and use it from WSO2 key Managers and gateways. So if we designed to have same application in store irrespective deployment it deployed we can achieve this requirement.

We have one API which can deployed in multiple environments. But can only one subscription for all environments and same set of tokens will work any environment. Still each environment have their own key manager and when it comes to validate again direct to same external key manager.

To understand more about API publisher publishing to multiple environment please refer this article.
https://docs.wso2.com/display/AM200/Maintaining+Separate+Production+and+Sandbox+Gateways

To study about deploying same API across multiple environments and URL to automatically resolve based on deployment refer this article.
http://sanjeewamalalgoda.blogspot.com/2016/07/how-to-define-environment-specific-url.html
http://nuwanzone.blogspot.com/2015/03/api-gateways-with-dedicated-back-ends.html

External key manager concept and more details.
https://docs.wso2.com/display/AM190/Configuring+a+Third-Party+Key+Manager

Display multiple gateway URLs in API store can achieve with external gateway concept.
Please refer following diagram for complete solution.


Wednesday, August 10, 2016

How to use Java scrip mediator to modify outgoing message body during mediation flow - WSO2 ESB/ API Manager

Here in this post we will see how we can use Java scrip mediator to modify outgoing message body during mediation flow.
Below is the sample API configuration i used for this sample. As you can see here i will extract password field from incoming message and send it as symbol to back end service. Since i need to check what actually pass to back end server i added TCPMon between gateway and back end server. That is why 8888 port is appear there. 
  <api name="TestAPI" context="/test">
      <resource methods="POST" url-mapping="/status" faultSequence="fault">
         <inSequence>
            <script language="js">var symbol = mc.getPayloadXML()..*::password.toString();
               mc.setPayloadXML(
                  &lt;m:getQuote xmlns:m="http://services.samples/xsd"&gt;
                     &lt;m:request&gt;
                        &lt;m:symbol&gt;{symbol}&lt;/m:symbol&gt;
                     &lt;/m:request&gt;
                  &lt;/m:getQuote&gt;);</script>
            <send>
               <endpoint name="test-I_APIproductionEndpoint_0">
                  <http uri-template="http://127.0.0.1:8888/"/>
               </endpoint>
            </send>
         </inSequence>
         <outSequence>
            <send/>
         </outSequence>
      </resource>
   </api>
Then i send request as follows. As you can see here password will have some data and i'm sending post request.
curl -v -X POST -H "Content-Type: application/json" -d '{"username":"xyz","password":"xyz & abc"}' http://127.0.0.1:8280/test/status
I can see my values pass to back end properly. Since i added TCPMon between ESB and back end to verify out going message i can see what going out from ESB. Also following is wore logs captured for my message.
[2016-08-10 14:30:21,236] DEBUG - wire >> "POST /test/status HTTP/1.1[\r][\n]"
[2016-08-10 14:30:21,236] DEBUG - wire >> "Host: 127.0.0.1:8280[\r][\n]"
[2016-08-10 14:30:21,236] DEBUG - wire >> "User-Agent: curl/7.43.0[\r][\n]"
[2016-08-10 14:30:21,236] DEBUG - wire >> "Accept: */*[\r][\n]"
[2016-08-10 14:30:21,236] DEBUG - wire >> "Content-Type: application/json[\r][\n]"
[2016-08-10 14:30:21,236] DEBUG - wire >> "Content-Length: 41[\r][\n]"
[2016-08-10 14:30:21,237] DEBUG - wire >> "[\r][\n]"
[2016-08-10 14:30:21,237] DEBUG - wire >> "{"username":"xyz","password":"xyz & abc"}"
[2016-08-10 14:30:21,245] DEBUG - wire << "POST /status HTTP/1.1[\r][\n]"
[2016-08-10 14:30:21,245] DEBUG - wire << "Content-Type: application/json[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "Accept: */*[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "Transfer-Encoding: chunked[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "Host: 127.0.0.1:8888[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "Connection: Keep-Alive[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "[\r][\n]"
[2016-08-10 14:30:21,246] DEBUG - wire << "2f[\r][\n]"
[2016-08-10 14:30:21,247] DEBUG - wire << "{"getQuote":{"request":{"symbol":"xyz & abc"}}}[\r][\n]"
[2016-08-10 14:30:21,247] DEBUG - wire << "0[\r][\n]"
[2016-08-10 14:30:21,247] DEBUG - wire << "[\r][\n]"

TCPMon output.













Tuesday, August 9, 2016

How to create axis2 service repository using WSO2 Governance Registry

Sometimes we need to store all service metadata in single place and maintain changes, life cycles etc. To address this we can implement this as automated process. In this example i will specifically focus on axis2 services and discuss how to create service repository for axis 2 services.

Here is the detailed flow.
  • In jenkins(or any other task scheduler to check available services frequentrly) we will deployed scheduled task to trigger some event periodically.
  • Periodic task will call WSO2 App Server’s admin services to get service metadata. To list service meta data for axis2 services we can call service admin soap service (https://127.0.0.1:9443/services/ServiceAdmin?wsdl)
  • In same way we can call all services and get complete service data(if you need other service types in addition to axis2 services).
  • Then we can call registry rest API and push that information.  Please refer this article for more information about Registry REST API.  

If we consider proxy service details then we can follow approach listed below.
Create web service client for https://127.0.0.1:9443/services/ServiceAdmin?wsdl service and invoke it from client. See following Soapui sample to get all proxy services deployed in ESB.



You will see response like below. There you will all details related to given proxy service such as wsdls, service status, service type etc. So you can list all service metadata using the information we retrieved from this web service call.


<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Body>
      <ns:listServicesResponse xmlns:ns="http://org.apache.axis2/xsd">
         <ns:return xsi:type="ax2356:ServiceMetaDataWrapper"
 xmlns:ax2356="http://mgt.service.carbon.wso2.org/xsd" 
xmlns:ax2358="http://neethi.apache.org/xsd" xmlns:ax2359="http://util.java/xsd" 
xmlns:ax2354="http://utils.carbon.wso2.org/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <ax2356:numberOfActiveServices>4</ax2356:numberOfActiveServices>
            <ax2356:numberOfCorrectServiceGroups>4</ax2356:numberOfCorrectServiceGroups>
            <ax2356:numberOfFaultyServiceGroups>0</ax2356:numberOfFaultyServiceGroups>
            <ax2356:numberOfPages>1</ax2356:numberOfPages>
            <ax2356:serviceTypes>axis2</ax2356:serviceTypes>
            <ax2356:serviceTypes>js_service</ax2356:serviceTypes>
            <ax2356:services xsi:type="ax2356:ServiceMetaData">
               <ax2356:active>true</ax2356:active>
               <ax2356:description xsi:nil="true"/>
               <ax2356:disableDeletion>false</ax2356:disableDeletion>
               <ax2356:disableTryit>false</ax2356:disableTryit>
               <ax2356:eprs xsi:nil="true"/>
               <ax2356:foundWebResources>false</ax2356:foundWebResources>
               <ax2356:mtomStatus xsi:nil="true"/>
               <ax2356:name>echo</ax2356:name>
               <ax2356:operations xsi:nil="true"/>
               <ax2356:scope xsi:nil="true"/>
               <ax2356:securityScenarioId xsi:nil="true"/>
               <ax2356:serviceDeployedTime>1970-01-01 05:30:00</ax2356:serviceDeployedTime>
               <ax2356:serviceGroupName>Echo</ax2356:serviceGroupName>
               <ax2356:serviceId xsi:nil="true"/>
               <ax2356:serviceType>axis2</ax2356:serviceType>
               <ax2356:serviceUpTime>17023day(s) 6hr(s) 16min(s)</ax2356:serviceUpTime>
               <ax2356:serviceVersion xsi:nil="true"/>
               <ax2356:tryitURL>http://172.17.0.1:9763/services/echo?tryit</ax2356:tryitURL>
               <ax2356:wsdlPortTypes xsi:nil="true"/>
               <ax2356:wsdlPorts xsi:nil="true"/>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/echo?wsdl</ax2356:wsdlURLs>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/echo?wsdl2</ax2356:wsdlURLs>
            </ax2356:services>
            <ax2356:services xsi:type="ax2356:ServiceMetaData">
               <ax2356:active>true</ax2356:active>
               <ax2356:description xsi:nil="true"/>
               <ax2356:disableDeletion>false</ax2356:disableDeletion>
               <ax2356:disableTryit>false</ax2356:disableTryit>
               <ax2356:eprs xsi:nil="true"/>
               <ax2356:foundWebResources>false</ax2356:foundWebResources>
               <ax2356:mtomStatus xsi:nil="true"/>
               <ax2356:name>HelloService</ax2356:name>
               <ax2356:operations xsi:nil="true"/>
               <ax2356:scope xsi:nil="true"/>
               <ax2356:securityScenarioId xsi:nil="true"/>
               <ax2356:serviceDeployedTime>1970-01-01 05:30:00</ax2356:serviceDeployedTime>
               <ax2356:serviceGroupName>HelloWorld</ax2356:serviceGroupName>
               <ax2356:serviceId xsi:nil="true"/>
               <ax2356:serviceType>axis2</ax2356:serviceType>
               <ax2356:serviceUpTime>17023day(s) 6hr(s) 16min(s)</ax2356:serviceUpTime>
               <ax2356:serviceVersion xsi:nil="true"/>
               <ax2356:tryitURL>http://172.17.0.1:9763/services/HelloService?tryit</ax2356:tryitURL>
               <ax2356:wsdlPortTypes xsi:nil="true"/>
               <ax2356:wsdlPorts xsi:nil="true"/>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/HelloService?wsdl</ax2356:wsdlURLs>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/HelloService?wsdl2</ax2356:wsdlURLs>
            </ax2356:services>
            <ax2356:services xsi:type="ax2356:ServiceMetaData">
               <ax2356:active>true</ax2356:active>
               <ax2356:description xsi:nil="true"/>
               <ax2356:disableDeletion>false</ax2356:disableDeletion>
               <ax2356:disableTryit>false</ax2356:disableTryit>
               <ax2356:eprs xsi:nil="true"/>
               <ax2356:foundWebResources>false</ax2356:foundWebResources>
               <ax2356:mtomStatus xsi:nil="true"/>
               <ax2356:name>Version</ax2356:name>
               <ax2356:operations xsi:nil="true"/>
               <ax2356:scope xsi:nil="true"/>
               <ax2356:securityScenarioId xsi:nil="true"/>
               <ax2356:serviceDeployedTime>1970-01-01 05:30:00</ax2356:serviceDeployedTime>
               <ax2356:serviceGroupName>Version</ax2356:serviceGroupName>
               <ax2356:serviceId xsi:nil="true"/>
               <ax2356:serviceType>axis2</ax2356:serviceType>
               <ax2356:serviceUpTime>17023day(s) 6hr(s) 16min(s)</ax2356:serviceUpTime>
               <ax2356:serviceVersion xsi:nil="true"/>
               <ax2356:tryitURL>http://172.17.0.1:9763/services/Version?tryit</ax2356:tryitURL>
               <ax2356:wsdlPortTypes xsi:nil="true"/>
               <ax2356:wsdlPorts xsi:nil="true"/>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/Version?wsdl</ax2356:wsdlURLs>
               <ax2356:wsdlURLs>http://172.17.0.1:9763/services/Version?wsdl2</ax2356:wsdlURLs>
            </ax2356:services>
         </ns:return>
      </ns:listServicesResponse>
   </soapenv:Body>
</soapenv:Envelope>


We can automate this service metadata retrieving process and persist it to registry. Please refer below diagram to understand flow for this use case. Discovery agent will communicate with servers and use REST client to push events to registry.

Untitled drawing(1).jpg

How to handle authentication failures from custom authentication handler - WSO2 API Manager

When we are implementing a custom Authentication Handler for wso2 API Manager we need to handle the case when authentication handler returns false. That mean authentication got failed and we need to send error back to client. Following is the sample code of the handleRequest method.
public boolean handleRequest(MessageContext messageContext) {
    try {
        if (authenticate(messageContext)) {
            return true;
        }
        else{
          //Your logic should go here.
        }
    } catch (APISecurityException e) {
        e.printStackTrace();
    }
    return false;
}

Ideally you need to call handleAuthFaliure method with message context.

private void handleAuthFailure(MessageContext messageContext, APISecurityException e) 

When you call that method please create APISecurityException object(as listed below) and pass it. Then error code and error messages will be picked from there and automatically send error to client(by default we return 401 for security exceptions and if defined then send defined error code and message).

public class APISecurityException extends Exception {
    private int errorCode;
    public APISecurityException(int errorCode, String message) {
        super(message);
        this.errorCode = errorCode;
    }
    public APISecurityException(int errorCode, String message, Throwable cause) {
        super(message, cause);
        this.errorCode = errorCode;
    }
    public int getErrorCode() {
        return errorCode;
    }
}
As you may already know you can generate error within your code and sent it back to client. But since API Manager have this capability you can easily use this without writing your own logic. And another advantage is if you pass error like this then it will go through auth_faliure_handler sequence and you can do any transformation there in a way it effect to all authentication failures.