How to pass Basic authntication headers to backend server via API Manager

First let me explain how authorization headers work in API Manager. When user send authorization header along with API request we will use it for API authentication purpose. And we will drop it from out going message.
If you want to pass clients auth headers to back end server without dropping them at gateway you can enable following parameter and disable it.
Update following property in /repository/conf/api-manager.xml and restart server.

false

Then it will not drop user sent authorization headers at gateway. So whatever user send will go to back end as well

Send API request with Basic Auth header.

Incoming message to API gateway. As you can see we do not use API Manager authentication here. For this we can set resource auth type as none when we create API. Then send Basic auth header that need to pass back end server.
[2015-02-27 18:08:05,010] DEBUG - wire >> "GET /test-sanjeewa1/1.0.0 HTTP/1.1[\r][\n]"
[2015-02-27 18:08:05,011] DEBUG - wire >> "User-Agent: curl/7.32.0[\r][\n]"
[2015-02-27 18:08:05,011] DEBUG - wire >> "Host: 10.100.1.65:8280[\r][\n]"
[2015-02-27 18:08:05,011] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-02-27 18:08:05,011] DEBUG - wire >> "Authorization: Basic 2690b6dd2af649782bf9221fa6188[\r][\n]"
[2015-02-27 18:08:05,011] DEBUG - wire >> "[\r][\n]"

Out going message from gateway. You can see client sent Basic auth header is present in out going message
[2015-02-27 18:08:05,024] DEBUG - wire << "GET http://localhost/apim1/ HTTP/1.1[\r][\n]"
[2015-02-27 18:08:05,025] DEBUG - wire << "Authorization: Basic 2690b6dd2af649782bf9221fa6188[\r][\n]"
[2015-02-27 18:08:05,025] DEBUG - wire << "Accept: */*[\r][\n]"
[2015-02-27 18:08:05,025] DEBUG - wire << "Host: localhost:80[\r][\n]"
[2015-02-27 18:08:05,025] DEBUG - wire << "Connection: Keep-Alive[\r][\n]"
[2015-02-27 18:08:05,026] DEBUG - wire << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
[2015-02-27 18:08:05,026] DEBUG - wire << "[\r][\n]"


Other possible option is setting Basic auth headers at API gateway. For this we have 2 options.

01. Define Basic auth headers in API when you create API(see attached image). 

In API implement phase you can provide required basic auth details. Then API manager gateway will send provided authorization details as basic oauth headers to back end. Here we can let client to send Bearer token authorization header with API request. And gateway will drop it(after Bearer token validation) and pass Basic auth header to back end.

Incoming message to API gateway. Here user send Bearer token to gateway. Then gateway validate it and drop from out message.
[2015-02-27 17:36:15,580] DEBUG - wire >> "GET /test-sanjeewa/1.0.0 HTTP/1.1[\r][\n]"
[2015-02-27 17:36:15,595] DEBUG - wire >> "User-Agent: curl/7.32.0[\r][\n]"
[2015-02-27 17:36:15,595] DEBUG - wire >> "Host: 10.100.1.65:8280[\r][\n]"
[2015-02-27 17:36:15,595] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-02-27 17:36:15,595] DEBUG - wire >> "Authorization: Bearer 2690b6dd2af649782bf9221fa6188-[\r][\n]"
[2015-02-27 17:36:15,595] DEBUG - wire >> "[\r][\n]"

Out going message from gateway. You can see Basic auth header added to out going message
[2015-02-27 17:36:20,523] DEBUG - wire << "GET http://localhost/apim1/ HTTP/1.1[\r][\n]"
[2015-02-27 17:36:20,539] DEBUG - wire << "Authorization: Basic YWRtaW46YWRtaW4=[\r][\n]"
[2015-02-27 17:36:20,539] DEBUG - wire << "Accept: */*[\r][\n]"
[2015-02-27 17:36:20,540] DEBUG - wire << "Host: localhost:80[\r][\n]"
[2015-02-27 17:36:20,540] DEBUG - wire << "Connection: Keep-Alive[\r][\n]"
[2015-02-27 17:36:20,540] DEBUG - wire << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"


02. This is also same as previous sample. But if need you can set API resource authorization type as none. Then client don't need to send anything in request. But APIM will add Basic auth headers to outgoing message.
You can understand message flow and headers by looking following wire log 

Incoming message to API gateway
[2015-02-27 17:37:10,951] DEBUG - wire >> "GET /test-sanjeewa/1.0.0 HTTP/1.1[\r][\n]"
[2015-02-27 17:37:10,953] DEBUG - wire >> "User-Agent: curl/7.32.0[\r][\n]"
[2015-02-27 17:37:10,953] DEBUG - wire >> "Host: 10.100.1.65:8280[\r][\n]"
[2015-02-27 17:37:10,953] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-02-27 17:37:10,953] DEBUG - wire >> "[\r][\n]"

Out going message from gateway. You can see Basic auth header is present in out going message
[2015-02-27 17:37:13,766] DEBUG - wire << "GET http://localhost/apim1/ HTTP/1.1[\r][\n]"
[2015-02-27 17:37:13,766] DEBUG - wire << "Authorization: Basic YWRtaW46YWRtaW4=[\r][\n]"
[2015-02-27 17:37:13,766] DEBUG - wire << "Accept: */*[\r][\n]"
[2015-02-27 17:37:13,766] DEBUG - wire << "Host: localhost:80[\r][\n]"
[2015-02-27 17:37:13,766] DEBUG - wire << "Connection: Keep-Alive[\r][\n]"

How to modify API Manager publisher to remove footer - API Manager 1.8.0

1. Go to publisher jaggery app (/repository/deployment/server/jaggeryapps/publisher)

2. Go to subthemes folder in publisher (site/themes/default/subthemes)

3. Create a folder with the name of your subtheme. For example "nofooter"

4. Create a folder called 'css' inside 'nofooter' folder

5. Copy the "/repository/deployment/server/jaggeryapps/publisher/site/themes/default/css/localstyles.css" to the new subtheme's css location " /repository/deployment/server/jaggeryapps/publisher/site/themes/default/subthemes/nofooter/css/"

6. Copy the "/repository/deployment/server/jaggeryapps/publisher/site/themes/default/images" folder to the new subtheme location " /repository/deployment/server/jaggeryapps/publisher/site/themes/default/subthemes/nofooter/"

7. add following css to localstyles.css file in "/repository/deployment/server/jaggeryapps/publisher/site/themes/default/subthemes/nofooter/css/" folder

#footer{
    display:none;
}

8. Edit "/repository/deployment/server/jaggeryapps/publisher/site/conf/site.json" file as below in order to make the new sub theme as the default theme.
         "theme" : {
               "base" : "default",
               "subtheme" : "nofooter"
        }

How monitor WSO2 server CPU usage and generate thread dump on high CPU usage using simple shell script

When we deployed WSO2 servers in production deployments we may need to monitor them for high CPU and memory usages. So in this article i will describe how we can use simple shell script to monitor server CPU usage and generate thread dump using jstack command.

First you need to create test.sh script using following command.

 vi test.sh 

Then paste following script content.

#!/bin/bash
# 1: ['command\ name' or PID number(,s)] 2: MAX_CPU_PERCENT
[[ $# -ne 2 ]] && exit 1
PID_NAMES=$1
# get all PIDS as nn,nn,nn
if [[ ! "$PID_NAMES" =~ ^[0-9,]+$ ]] ; then
    PIDS=$(pgrep -d ',' -x $PID_NAMES)
else
    PIDS=$PID_NAMES
fi
#  echo "$PIDS $MAX_CPU"
MAX_CPU="$2"
MAX_CPU="$(echo "($MAX_CPU+0.5)/1" | bc)"
LOOP=1
while [[ $LOOP -eq 1 ]] ; do
    sleep 0.3s
    # Depending on your 'top' version and OS you might have
    #   to change head and tail line-numbers
    LINE="$(top -b -d 0 -n 1 -p $PIDS | head -n 8 \
        | tail -n 1 | sed -r 's/[ ]+/,/g' | \
        sed -r 's/^\,|\,$//')"
    # If multiple processes in $PIDS, $LINE will only match\
    #   the most active process
    CURR_PID=$(echo "$LINE" | cut -d ',' -f 1)
    # calculate cpu limits
    CURR_CPU_FLOAT=$(echo "$LINE"| cut -d ',' -f 9)
    CURR_CPU=$(echo "($CURR_CPU_FLOAT+0.5)/1" | bc)
    echo "PID $CURR_PID: $CURR_CPU""%"
    if [[ $CURR_CPU -ge $MAX_CPU ]] ; then
        now="$(date)"
        echo "PID $CURR_PID ($PID_NAMES) went over $MAX_CPU""%" on $now
        jstack $CURR_PID > ./$now+jlog.txt
        echo "[[ $CURR_CPU""% -ge $MAX_CPU""% ]]"
        LOOP=0
        break
    fi
done
echo "Stopped"

Then we need to get process id of running WSO2 server by running following command.

sanjeewa@sanjeewa-ThinkPad-T530:~/work$ jps
30755 Bootstrap
8543 Jps
4892 Main

Now we know carbon server running with process ID 30755. Then we can start our script by providing init parameters(process ID and CPU limit). So it will keep printing CPU usage in terminal and once it reached limit it will take thread dump using jstack command. It will create new file with with embedding current date time and push Jstack output to it.

We can start scritp like this.

 sh test.sh <processId> <CPU Limit>

sanjeewa@sanjeewa-ThinkPad-T530:~/work$ sh test.sh 30755 10
PID 30755: 0%
PID 30755: 0%
PID 30755: 0%
PID 30755: 0%
PID 30755 (30755) went over 10% on 2015 පෙබරවාරි 19 වැනි බ්‍රහස්පතින්දා 14:44:55 +0530
[[ 13% -ge 10% ]]
Stopped

As you can see when CPU goes above 10% it will create log file and append thread dump.

How to add sub theme to API Manager jaggery applications - API Manager store/publisher customization

In API Manager we can add sub themes and change look and feel of jaggery applications. Here in this post i will provide high level instructions for that. Customize the existing theme and, add the new theme as a "sub theme" to the store and publisher.

(1) Navigate to "/repository/deployment/server/jaggeryapps/store/site/themes/fancy/subthemes" directory.
(2) Create a directory with the name of your subtheme. For example "test".
(3) Copy the "/repository/deployment/server/jaggeryapps/store/site/themes/fancy/css/styles-layout.css" to the new subtheme location "repository/deployment/server/jaggeryapps/store/site/themes/fancy/subthemes/test/css/styles-layout.css".
(4) At the end of the copied file add the css changes in [a].
(5) Edit "/repository/deployment/server/jaggeryapps/store/site/conf/site.json" file as below in order to make the new sub theme as the default theme.
        "theme" : {
               "base" : "fancy",
               "subtheme" : "test"
        }



Add custom css to new sub theme

.link-to-user {
  max-width: 100px;
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
}
@media only screen and (max-width:1200px){
    .navbar .nav.pull-right.login-sign-up{
        width: 100%;
        text-align:right;
float:left;
border-left:solid 1px #292e38;
border-bottom:solid 1px #292e38;
    }
.menu-content .navbar .nav.pull-right.login-sign-up > li{
        float:left;
 }
 .menu-content .navbar .nav.pull-right.login-sign-up > li.dropdown > a , .menu-content .navbar .nav.pull-right.login-sign-up > li.dropdown.open > a{
        color:#000 !important;
 }
 .search-section-custom{
         margin-top:62px;
 }
 #wrap > div.header + div.clearfix + .container-fluid{
padding-right:0;
 }
}

WSO2 API Manager - Troubleshoot common deployment issues

 Here are some of useful tips when you work on deployment related issues.

-Dsetup doesn't work for usage tables. Usage table definitions are not there in setup scripts.
In WSO2 API Manager and BAM deployment we need user manager database, registry database and api manager database. We do have setup scripts for those database under db scripts folder of product distribution.  There is no need to create any tables in stats database(manually or using setup script) as API Manager toolbox(deployed in BAM) will create them when hive queries get executed. -Dsetup option will not apply to the hive scripts inside toolbox deployed in BAM.

Understand connectivity between components in distributed API manager deployment.
This is important when you work on issues related to distributed API Manager deployment. Following steps to explain connectivity between components. It would be useful to listed them here.

1. Changed the admin password
2. Tried to log in to publisher and got the insufficient privilege error
3. Then changed the admin password in authManager element in api-manager.xml
4. Restarted and I was able to login to API publisher. Then I created an API and tried to publish. Got a permission error again.
5. Then, I changed password under API Gateway element in api-manager.xml
6. Restarted and published the API. Then, tried to invoke an API using an existing key. Got the key validation error.
7. Then, I changed the admin password in KeyManager element in api-manager.xml and all issues got resolved.

Thrift key validation does not work when we have load balancer fronted key manager.
Reason for this is most of load balancers not capable of routing traffic in session aware manner. So in such cases its always recommend to use WS key validation client.

Usage data related issues.
When you work with usage data related issues first we should check data source configurations in BAM and API Manager. Then we need to check created tables in usage database. Most of reported issues are due to configuration issues.  Same applies to billing sample as well.

WSO2 API Manager - How to customize API Manager using extension points

Here in this article i will discuss about common extensions available in API Manager and how we can use them.

• In API manager we have provided many extension points to extend functionalities based on custom requirements.
• Workflow
• JWT generation(http://wso2.com/library/articles/2014/12/customize-json-web-token-generation-with-wso2-api-manager-1.8.0/)
• Claim retrieve
• Adding new handlers
• Extension handlers
• Sequence support.
• Mediation extension(https://docs.wso2.com/display/AM160/Adding+a+Mediation+Extension)\
• How to pass 2 auth headers in same request(one for API Manager and one for backend).(http://sanjeewamalalgoda.blogspot.com/2014/11/how-to-use-custom-authentication-header.html)
• Enable mutual SSL for server to server communication. We can use this for API Manager gateway to key manager validation call.(http://sanjeewamalalgoda.blogspot.com/2014/11/how-to-enable-mutual-ssl-connection.html)

WSO2 API Manager visibiity, subscription availability and relation between them

When we create APIs we need to aware about API visibility and subscriptions. Normally API visibility directly couple with subscription availability(simply because you cannot subscribe to something you dont see in store). See following diagram for more information about relationship between them.

Visibility - we can contorl how other users can see our APIS

Subscription availability - How other users can subscribe to APIs created by us