How to avoid getting incorrect access tokens due to constraint violation when we have high load on token API(CON_APP_KEY violated).

Sometimes you may see following behavior when we have very high load on token API.
1. Call https://localhost:8243/token
2. Get constraint error.

{org.wso2.carbon.identity.oauth2.dao.TokenPersistenceTask} - Error occurred while persisting access token bsdsadaa209esdsadasdae21a17d {org.wso2.carbon.identity.oauth2.dao.TokenPersistenceTask}
org.wso2.carbon.identity.oauth2.IdentityOAuth2Exception: Access Token for consumer key : H5sadsdasdasddasdsa, user : sanjeewa and scope : default already exists
at org.wso2.carbon.identity.oauth2.dao.TokenMgtDAO.storeAccessToken(TokenMgtDAO.java:194)
at org.wso2.carbon.identity.oauth2.dao.TokenMgtDAO.persistAccessToken(TokenMgtDAO.java:229)
at org.wso2.carbon.identity.oauth2.dao.TokenPersistenceTask.run(TokenPersistenceTask.java:56)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.sql.SQLIntegrityConstraintViolationException: ORA-00001: unique constraint (WSO2_APIM.CON_APP_KEY) violated

3. Attempt to use the access token when calling an API but get HTTP Status-Code=401 (Unauthorized) Invalid Credentials error

This issue happens because our token storing logic is not blocking call(this was implemented as improvement to token API as persisting can block token generation flow).
So due to that we already returned incorrect token to client(which is not already persisted). This happens only if constraint failed when we try to persist token.
But at that time we may return token to client.

If we made token persisting pool size 0 then this issue will not be there and user will immediately get error (probably internal server error) and token will not be return to client.
See following code block

try {
tokenMgtDAO.storeAccessToken(accessToken, oAuth2AccessTokenReqDTO.getClientId(),
accessTokenDO, userStoreDomain);
} catch (IdentityException e) {
throw new IdentityOAuth2Exception(
"Error occurred while storing new access token : " + accessToken, e);
}

You can set pool size as follows. By default it set to 100.
wso2am-1.9.0/repository/conf/identity.xml

<JDBCPersistenceManager>
    <SessionDataPersist>
        <PoolSize>0</PoolSize>
    </SessionDataPersist>
</JDBCPersistenceManager>

This Will resolve your issue