Error: Could not find any functions to execute for transaction - hyperledger-fabric

my issue is that I am defining transaction in model file and then using it in js script but it throws an error " Error: Could not find any functions to execute for transaction." when i try to execute.It occurs during testing of the code
my mode file
/**
* New model file
*/
/**
* New model file
*/
namespace org.acme.bank
participant accountholder identified by bankid
{
o String bankid
o String firstname
o String lastname
o String address
}
asset acount identified by accno
{
o String accno
o String balance
-->accountholder customer1
}
transaction amountTransfer
{
o String tid
o String amount
-->acount owner1
-->acount owner2
}
my script.js
/**
* Track the trade of a commodity from one trader to another
* #param {org.acme.bank.amountTransfer} Transfer - to trade
* #transactiton
*/
function Transfer(Transfer)
{
var amount1=Transfer.owner1.balance
var amount2=Transfer.owner2.balance
if(Transfer.amount>amount1)
{
return 0;
}else
{
owner1.balance-=Transfer.amount
owner2.balance+=Transfer.amount
return getAssetRegistry('org.acme.bank.acount')
.then(function (assetRegistry) {
return assetRegistry.update(Transfer.owner1);
}).then(function () {
return getAssetRegistry('org.acme.bank.acount');
}).then(function (assetRegistry) {
return assetRegistry.update(Transfer.owner2);
});
}
}
thank you in advance

I found a tiny typo #transactiton in my script.js and I changer it then the error no longer occurs.
I think the code below works as you expected.
/**
* Track the trade of a commodity from one trader to another
* #param {org.acme.bank.amountTransfer} Transfer - to trade
* #transaction
*/
function Transfer(Transfer)
{
var amount1=Transfer.owner1.balance
var amount2=Transfer.owner2.balance
if(Transfer.amount>amount1)
{
return 0;
}else
{
var owner1 = Transfer.owner1
var owner2 = Transfer.owner2
owner1.balance-=Transfer.amount
owner2.balance+=Transfer.amount
return getAssetRegistry('org.acme.bank.acount')
.then(function (assetRegistry) {
return assetRegistry.update(owner1);
}).then(function () {
return getAssetRegistry('org.acme.bank.acount');
}).then(function (assetRegistry) {
return assetRegistry.update(owner2);
});
}
}
Now balance and amount field's type in your model are changed to Integer.

Related

Hyperledger Composer Playground Execution Error

I'm trying to create my own crypto currency-'WhizCoin' in Hyperledger Composer Playground.
I have created basic model and script files. 2 Participants- Sender and Receiver, and 1 Asset- Coins, and 1 Transaction- DistributeCoins.
/* Model.cto File:- */
namespace org.whiz.whizcoin
// ASSETS:-
abstract asset Currency{
o Double rate default=100.00
o Double coins
}
asset WhizCoin identified by currencyid extends Currency{
o String currencyid default = "WhizCoin"
//--> Member owner
}
// PARTICIPANTS:-
abstract participant user{
o String name
o Integer balance
}
participant Member identified by memberId extends user {
o String memberId
}
// Transactions:-
transaction distributeCoins {
--> Member receiver
o Double amount
-->Currency wc
}
/* logic.js File:- */
'use strict';
var NS = 'org.whiz.whizcoin';
/**
* #param {org.whiz.whizcoin.distributeCoins} distributeCoins - Distribute Coins
* #transaction
*/
function distributeCoins(distributeCoins) {
distributeCoins.receiver.balance += distributeCoins.amount;
distributeCoins.wc.coins -= distributeCoins.amount;
return getParticipantRegistry('org.whiz.whizcoin.Member')
.then(function (ParticipantRegistry) {
return ParticipantRegistry.update(distributeCoins.receiver);
})
.then(function() {
return getAssetRegistry('org.whiz.whizcoin.WhizCoin');
})
.then(function (AssetRegistry) {
return AssetRegistry.update(distributeCoins.whizcoin);
});
}
After submitting the transaction, the error I'm getting is:
Error: Expected a Resource or Concept.
I'm really stuck and couldn't find any solution anywhere else. Please help. Thanks in advance!
In the section of your logic where you are updating the Asset Registry you are using a variable that doesn't exist in the transaction, whizcoin. In your model file you use the name wc for this. Replacing AssetRegistry.update(distributeCoins.whizcoin) with AssetRegistry.update(distributeCoins.wc) should fix your issue.

Easy way to handle nested transactions

Suppose there is an "addUser" function, inside we need to insert a record to "Account" table and "User" table, so the two steps have to be within a transaction too, so we will write the following code:
function addUser (userName, password) {
sequelize.transaction(function () {
return AccountModel.create(...)
.then(UserModel.create(...))
})
}
However, in another "addTeam" function, inside we need to insert a record to "Team" table and create a admin user using the above function. The function also need to be wrapped inside a transaction.
So the problem comes, the "addUser" function sometimes need to begin a new transaction, and sometimes need to use the transaction passed in. The most obvious ways is below:
function addUser (userName, password, transaction) {
let func = function (t) {
return AccountModel.create(..., t)
.then(()=>UserModel.create(..., t)));
if (transaction) func(t);
else sequelize.transaction(x=>func(t));
}
function addTeam() {
sequelize.transaction(x=> {
TeamModel.create(..., x)
.then(y=>addUser(x));
});
}
Obviously, it is awful. How to deal with it easily, which let transaction totally transparent to the caller like below:
#Transaction
async function addUser(userName, password) {
await AccountModel.create(...);
await UserModel.create(...);
}
#Transaction
async function addTeam(...) {
await TeamModel.create(...);
await addUser(...);
}
sequelize.transaction accepts an options object - If options.transaction is set, this will create a savepoint in the transaction (provided that the SQL dialects supports it), otherwise it will create a new transaction
http://docs.sequelizejs.com/en/latest/api/sequelize/#transactionoptions-promise
So you should be able to do simply
sequelize.transaction({ transaction }, x=>func(t));
If you use CLS, a very simple helper function will do the work.
const cls = require('continuation-local-storage');
const Sequelize = require('sequelize');
const NAMESPACE = 'your-namespace';
// Use CLS for Sequelize
Sequelize.cls = cls.createNamespace(NAMESPACE);
const sequelize = new Sequelize(...);
/* * * * * * * * * * * * * * * * * * * * *
* THE MAGIC: Create a transaction wrapper
* * * * * * * * * * * * * * * * * * * * */
function transaction(task) {
return cls.getNamespace(NAMESPACE).get('transaction') ? task() : sequelize.transaction(task);
};
/* * * * * * * * * * * * * * * * * * * * *
* Your code below
* * * * * * * * * * * * * * * * * * * * */
function addUser(userName, password) {
return transaction(function() {
return AccountModel
.create(...)
.then(() => UserModel.create(...));
});
}
function addTeam() {
return transaction(function() {
return TeamModel
.create(...)
.then(() => addUser(...));
});
}
I solved the problem, and feel great.
I used the CLS feature sequelize provides, like the code below:
let namespace = Sequelize.cls = cls.createNamespace('myschool');
export const db = new Sequelize(config.db.url);
export const trans = option => operation => async function () {
let t = namespace.get('transaction');
let hasTrans = !!t;
t = t || await db.transaction();
try {
let result = await operation.apply(null, arguments);
if (!hasTrans) await t.commit();
return result;
}
catch (e) {
if (!hasTrans) await t.rollback();
throw e;
}
};
The above code just create a transaction and commit it if there is no transaction in the local context, otherwise just leave it.
And each business function want a transaction just need to use the above high order function to wrap like below:
export const createSchool = trans()( async (name, accountProps) => {
let school = await SchoolModel.create({name});
let teacher = await createTeacher({...accountProps, schoolId: school.get('id')});
return {school, teacher};
});
Although there is an accepted answer, below answer refers to a chain of commands (not necessarily nested) we may want to execute in the same transaction, using async-await:
try{
// create a transaction
let transaction = await sequelize.transaction();
// "parent" insertion, id is auto increment, so we will need the id of inserted data
let record = await Parent_Record.create({
data_1: req.params.data1,
data_2: req.params.data2,
....
}, {transaction});
//the array contains data depending upon the id of inserted parent transaction
for ( x in dataForChildTable) {
result = await Child_Record.create({
id: record.id, //same transaction so we can use record.id of parent
.....
other_data: dataForChildTable[x].otherData}, {transaction});
}
//commit the transaction
await transaction.commit();
} catch (err) {
console.log(err.message);
}
You could use zb-sequelize which adds transactional decorators.
import { Transactional, Tx } from 'zb-sequelize';
#Transactional
function addUser(user, password, #Tx transaction) {
// no need to create, commit or rollback a transaction.
}
You do need to initialize it with your sequelize instance.
import { Sequelize } from 'sequelize';
import { initSequelizeResolver } from 'zb-sequelize';
// you already have this somewhere.
const sequelize = new Sequelize(options);
// you need to add this:
initSequelizeResolver((args) => sequelize);

How to overload mongoose model instance in nodejs & Typescript

I'm trying to change the save() method, but I don't find where I can overload it. I use typescript and node.js.
For the moment, I have a UserModel that contains a mongoose.Schema and a mongoose.Model.
When I call UserModel.getModel() I retrieve the mongoose.Model from the UserModel.
I basically use a DAO to retrieve the Model class object.
user = message.getDataByKey('user');
user.save(function(err, data) {
// do stuff
});
I want to overload automatically the user object to use my own .save() method to check if there is error and always handle them by the same way.
When I set the Model, I do it like that:
public static model: any = model.Models.MongooseModel.getNewInstance(UserModel.modelName, UserModel._schema);
And in the parent:
public static getNewInstance(modelName, schema){
var Model: any = mongoose.model(modelName, schema);
// Overload methods.
//console.log(new Model());
// Return overloaded Model class.
return Model;
}
I would like to know if there is any way to overload the Model to make sure that each new instance from it will have my own .save method.
I thought use the statics/methods (methods actually, I guess) but it's empty or I know that the final object will have save/remove/update methods. So I don't know why it's not already into the object, I tried to console.log(Model and new Model()) but no save() method.
So I'm a little desappointed, maybe I missed something.
The fact is, I can't update directly the new Model() because they will be created later, in another context, I need to update the Model directly to make sure that the new instance from this model will have my extra function.
And I don't want to rewrite the basic .save() method, I just want to overload it to add extra validation.
Any idea? I'm kinda lost here, it's not that easy. Thx.
I found a solution to do so, I'm using typescript so I'll post both .ts and .js to everybody understand.
I use CommonJs compilation
Model.ts (Super model, parent of all models)
///<reference path='./../../lib/def/defLoader.d.ts'/>
/**
* Package that contains all Models used to interact with the database.
* TODO Use options http://mongoosejs.com/docs/guide.html
*/
export module Models {
/**
* Interface for all Models, except the parent class.
*/
export interface IModel{
/**
* Name of the model.
* It's a helper to always get the name, from instance or static.
* MUST start by uppercase letter!
*/
modelName: string;
/**
* Contains the static value of the public schema as object.
* It's a helper to always get the schema, from instance or static.
*/
schema: mongoose.Schema;
/**
* Contains the static value of the object used to manipulate an instance of the model.
* It's a helper to always get the model, from instance or static.
*/
model: any;
}
/**
* Parent class for all models.
* A model contains a mongoose schema and a mongoose model and other things.
*/
export class Model{
/**
* Suffix used to load automatically models.
*/
public static suffix: string = 'Model';
/**
* Suffix used to load automatically models.
* It's a helper to always get the schema, from instance or static.
*/
public suffix: string;
/**
* Name of the model.
* MUST start by uppercase letter!
*/
public static modelName: string = '';
/**
* Readable schema as object.
*/
public static schema: any;
/**
* Schema as mongoose Schema type.
*/
public static Schema: mongoose.Schema;
/**
* The mongoose model that uses the mongoose schema.
*/
public static model: any;
/**
* Use static values as instance values.
*/
constructor(){
// Use static values as instance values.
this.suffix = Model.suffix;
}
/**
* Returns a new mongoose.Schema customized instance.
* #param ChildModel Child model that made the call.
* #returns {*}
* #see http://mongoosejs.com/docs/2.7.x/docs/methods-statics.html
*/
public static getNewSchemaInstance(ChildModel): mongoose.Schema{
var schema: any = new mongoose.Schema(ChildModel.schema, {collection: ChildModel.modelName.toLowerCase()});
// Overload methods.
//schema.methods.toObject = function(callback){}
// Return overloaded instance.
return schema;
}
/**
* Retrieves a new Model instance and overload it to add statics methods available for all Models.
* #param ChildModel
* #returns {*}
* #see http://mongoosejs.com/docs/2.7.x/docs/methods-statics.html
*/
public static getNewModelInstance(ChildModel): any{
// Get the Model class.
var Model: any = mongoose.model(ChildModel.modelName, ChildModel.Schema);
/**
**************************************************************************************************
************************ Extended Model static methods for all Models ****************************
**************************************************************************************************
*/
/**
* Handler for all database/mongoose errors.
* #param err Error.
* #param data Data. Contains the model and the emitter. (+ more)
* #param callback Callback function to execute.
*/
Model.errorHandler = (err: any, data: any, callback: (message: (any) => any) => any) => {
// Extract data.
var _Model = data.model;
var __function = data.__function;
var __line = data.__line;
// Will contains the error.
var message:any = [];
// Mongo error.
if(err && err.name && err.name == 'MongoError'){
var _err = MongoError.parseMongoError(err);
if(err.code == 11000){
// Duplicate key on create.
message[0] = '__19';
message[1] = [_err.value, _err.field];
}else if(err.code == 11001){
// Duplicate key on update.
message[0] = '__20';
message[1] = [_err.value, _err.field];
}else{
// Non-managed mongo error.
if(dev()){
// Return not only the message but also some information about the error.
message[0] = [];
// Message. [0][1] could be args.
message[0][0] = '__21';
// Data.
message[1] = {
err: err,
model: _Model.modelName
};
}else{
message = '__21';
}
}
fs.appendFile(__config.path.base + __config.mongo.error.log, new Date() + ': ' + JSON.stringify({error: err, model: _Model.modelName, _err: _err}) + '\n');
}else if(err && err.name && err.name == 'ValidationError'){
// Validation error from mongoose.
var _err = MongoError.parseValidationError(err);
message[0] = [];
// Message. [0][1] could be args.
message[0][0] = '__24';
message[0][1] = [_err[0].value, _err[0].field, _err[0].type];
if(dev()){
// Will be send as args but not displayed in the message.
message[1] = {
err: _err,
model: _Model.modelName
};
}
fs.appendFile(__config.path.base + __config.mongo.error.log, new Date() + ': ' + JSON.stringify({error: err, model: _Model.modelName, _err: _err}) + '\n');
}else{
// Another error? I don't know if that could happens, but manage it anyway.
message[0] = '__22';
if(dev()){
message[1] = [err, _Model.modelName];// Will be send as args but not displayed in the message.
}
fs.appendFile(__config.path.base + __config.mongo.error.log, new Date() + ': ' + JSON.stringify({error: err, model: _Model.modelName}) + '\n');
}
callback(message);// return an error.
};
/**
* Check if the object exists and returns it in this case.
* #param object Object to find.
* #param callback Callback to execute.
* #return
* err Error if it happens. [null]
* found Found object or false.
*/
Model.exists = (object, callback): any => {
// If object is null or false or empty or whatever, don't do the research, the result could be wrong!
if(!object){
callback (null, false);
}else{
Model.findOne(object, function (err, found) {
if (err){
Model.errorHandler(err, ChildModel, callback);
}else if (found){
callback(null, found);
}else{
callback (null, false);
}
});
}
};
// Return overloaded instance.
return Model;
}
}
/**
* Class that manage MongoDb errors, used statically.
*/
export class MongoError{
/**
* Parse a mongo error to returns data from it because Mongo returns really bad errors.
* #param err The mongo error.
* #returns {*}
*/
public static parseMongoError(err): any{
var _err: any = {};
var _message: string = err.err;
if(err.code == 11000 || err.code == 11001){
var message = _message.split(':');
// Get the table where the error was generated.
_err.table = message[1].split('.')[1];
// Get the field name where the error was generated.
_err.field = message[1].split('.')[2].split(' ')[0].replace('$', '');
_err.field = _err.field.substr(0, _err.field.lastIndexOf('_'));
// Get the
_err.value = message[3].split('"')[1].replace('\\', '');
}
return _err;
}
/**
* Parse a mongoose validation error, probably generated during a save/update function.
* #param err The mongoose error.
* #returns {*}
*/
public static parseValidationError(err): any{
var _errors: any = new Array();
var i = 0;
for(var error in err.errors){
_errors[i] = [];
_errors[i]['field'] = err.errors[error]['path'];
_errors[i]['value'] = err.errors[error]['value'];
_errors[i]['type'] = err.errors[error]['type'];
i++;
}
return _errors;
}
}
}
The JS version:
http://pastebin.com/xBTr1ZVe
Error messages (__21, etc.) are:
"__19": "Unable to add the element, the value **_$0** for the field _$1 already exists, it cannot be duplicated.",
"__20": "Unable to update the element, the value **_$0** for the field _$1 already exists, it cannot be duplicated.",
"__21": "Unable to execute the requested operation. Database error 21. Please report to an administrator.",
"__22": "Unable to execute the requested operation. Database error 22. Please report to an administrator.",
"__23": "Validation error, the requested operation was aborted. Please check that all your data are correct and retry. Please report to an administrator. (code 23)",
"__24": "Unable to perform the operation, the value ***_$0** for the field _$1 didn't pass successfully the validation. Error: _$2",
Basically all my models should manage by themself these exception, of course. But if I forgot to do it, I'll get a managed exception, better.
Now I'll post a real Model, UserModel inheriting the parent Model.
///<reference path='./../../lib/def/defLoader.d.ts'/>
import model = require('./Model');
export module Models {
/**
* Model used to manage users.
* The model is primary static, but, to make it easy to use, some things are also stored for each instance.
* That allows the code to use both Model or instance of Model such as:
* Model.schema
* model.Schema
*/
export class UserModel extends model.Models.Model implements model.Models.IModel{
/**
*************************************************************************************************
****************************** Public methods & attributes **************************************
*************************************************************************************************
*/
/**
* Name of the model.
* MUST start by uppercase letter!
*/
public static modelName: string = 'User';
/**
* Readable schema as object.
*/
public static schema: any = require('../schemas/userSchema.js');
/**
* Schema as mongoose Schema type.
*/
public static Schema: mongoose.Schema = model.Models.Model.getNewSchemaInstance(UserModel);
/**
* The mongoose Model that uses the mongoose schema.
*/
public static model: any = model.Models.Model.getNewModelInstance(UserModel);
/**
* Helpers to always get the property, from instance or static.
*/
public modelName: string = UserModel.modelName;
public schema: mongoose.Schema = UserModel.schema;
public model: mongoose.Model<any> = UserModel.model;
/**
*************************************************************************************************
***************************** Extended methods & attributes **************************************
*************************************************************************************************
*/
/**
* These fields are protected, the user password is required to access to them.
* These fields are basically shared between applications.
* #private
*/
private static _protectedFields: string[] = [
'login',
'email'
];
/**
* Method to use to hash the user password.
*/
private static _passwordHashMethod: string = 'sha256';
/**
* Digest to use to hash the user password.
*/
private static _passwordDigest: string = 'hex';
/**
* Returns the protected fields.
* #returns {string[]}
*/
public static getProtectedFields(): string[]{
return this._protectedFields;
}
/**
* Hash a user password depending on the password hash configuration. Currently SHA256 in hexadecimal.
* Assuming crypto is global.
* #param password User password.
* #returns {string} Hashed password.
*/
public static hashPassword(password: string): string{
return crypto
.createHash(UserModel._passwordHashMethod)
.update(password)
.digest(UserModel._passwordDigest)
}
}
/**
* Don't forget that some methods such as exists() are written in the Model class and available for all Models.
* The following methods belong ONLY to the mongoose model instance, not to the Model class itself!
*
*************************************************************************************************
******************************** Extended Model methods *****************************************
*************************************************************************************************
*/
/**
* Connect a user to the game.
* #param user User to check. {}
* #param callback Callback to execute.
*/
UserModel.model.checkAuthentication = (user, callback) => {
// Force to provide login and password.
UserModel.model.exists({login: user.login, password: UserModel.hashPassword(user.password)}, function(err, userFound){
// Load public profile.
UserModel.model._getProtectedInformation(userFound, function(userPublic){
// Provides only public fields.
callback(new __message("__17", {err: err, user: userPublic}, !err && userFound ? true: false));
});
});
};
/**
* Get the protected fields for the found user.
* #param user User to find.
* #param callback Callback to execute.
*/
UserModel.model.getProtectedInformation = (user, callback) => {
// We are looking for an unique user.
UserModel.model.exists(user, function(err, userFound){
if(err){
UserModel.model.errorHandler(err, UserModel, callback);
}else{
// Load public profile.
UserModel.model._getProtectedInformation(userFound, function(userPublic){
// Provides only public fields.
callback(new __message('', {err: err, user: userPublic}, err ? false: true));
});
}
});
};
/**
* Get the protected fields of a user.
* #param user Instance of model.
* #param callback Callback to execute.
* #private
*/
UserModel.model.hashPassword = (user, callback): any => {
var err = false;
if(user && user.password){
user.password = UserModel.hashPassword(user.password);
}else{
err = true;
}
callback(new __message(err ? '__18': '', {user: user}, err ? false: true));
};
/**
*************************************************************************************************
*************************** Methods to use only locally (private) *******************************
*************************************************************************************************
*/
/**
* Get the protected fields of a user.
* #param user Instance of model.
* #param callback Callback to execute.
* #private
*/
UserModel.model._getProtectedInformation = (user, callback): any => {
var userPublic = {};
// Get fields to share.
var publicFields = UserModel.getProtectedFields();
// Fill the userPublic var with public fields only.
for(var field in publicFields){
userPublic[publicFields[field]] = user[publicFields[field]];
}
callback(userPublic);
};
}
The JS version:
http://pastebin.com/0hiaMH25
The schema:
/**
* Schema ued to create a user.
* #see http://mongoosejs.com/docs/2.7.x/docs/schematypes.html
*/
module.exports = userSchema = {
/**
* User Login, used as id to connect between all our platforms.
*/
login: {
type: 'string',
//match: /^[a-zA-Z0-9_-]{'+userSchema.login.check.minLength+','+userSchema.login.check.maxLength+'}$/,
trim: true,
required: true,
notEmpty: true,
unique: true,
check: {
minLength: 4,
maxLength: 16
}
},
/**
* User email.
*/
email: {
type: 'string',
lowercase: true,
match: /^[a-zA-Z0-9._-]+#[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$/,
required: true,
notEmpty: true,
unique: true,
check: {
minLength: 6,
maxLength: 30
}
},
/**
* User private password, the one hashed in SHA512 and stored on the database.
*/
password: {
type: 'string',
required: true,
check: {
length: 128
}
},
/**
* Salt to use to decrypt the password.
*/
passwordSalt: {
type: 'string',
check: {
length: 64
}
},
/**
* Password sent from user interface but hashed before be send on the network.
* Used to basically connect an user or generate the final password.
* Not stored in the DB.
*/
passwordProtected: {
type: 'string',
check: {
length: 64
}
},
/**
* Password wrote by the user on the GUI, not hashed or encrypted.
* Will be encrypted to respect the "passwordProtected" rules.
* Not stored in the DB.
*/
passwordPublic: {
type: 'string',
check: {
minLength: 8,
maxLength: 25
}
},
/**
* User banned status (Temporary of Definitive)
*/
banned: {
temporary : {
type : "number",
default : Date.now
},
definitive: {
type: 'boolean',
default: false
}
},
/**
* User right
*/
right : {
admin : {
type : "boolean",
default : false,
required: true
},
moderator : {
type : "boolean",
default : false,
required: true
}
}
};
So, what the code does?
Basically, in the Model.getNewModelInstance() I bind to the created model the errorHandler method that I will call if I found a DB error in the controller.
**UserController.js**
User.exists({email: user.email}, function(err, emailFound){
// If we got an err => Don't find couple User/pass
if (err) {
User.errorHandler(err, {model: User, __filename: __filename,__function: __function || 'subscription#exists', __line: __line}, function(err){
res.json(__format.response(err));
});
)
});
The __filename and so on are global functions that I use to get the current data, useful to debug. I'm still looking for a way to add this automatically but so far I couldn't. The __function doesn't exists when the function is anonymous. But it helps me to debug.
Any suggestion? That's a lot of piece of code.

How to retrieve more than 100 messages from the history of a PubNub channel?

The page about the PubNub History API states that
The history() function returns a list of up to 100 messages, the start
time token and the ending time token.
Is there a way to retrieve more than the 100 messages?
I'm currently not a paying customer of PubNub.
PubNub Load History More than 100 Messages
Sometimes you want to slice back in time over a linear stream of data. And often you'll want to do this at different levels of granularity. That is why PubNub Storage and Playback APIs provide maximum level of flexibility. However sometimes it ends up being a bit tricky to load data with the preferred result set.
PubNub Real-Time Network Storage and Playback
There are several considerations you may be seeking when loading transaction history over timelines that can potentially span millions of message in the transaction set. There are some great options available to you and we will cover two of them right now. The examples will be coded in JavaScript. The first example loads a summary of the data by grabbing the snapshots for the beginning of each hour for the past 24 hours. The second example shows you how to load all transactions in full detail and maximum granularity.
All Reference Files can be found on this GIST: Loading History from PubNub Mt.Gox Trades
Example PubNub Mt.Gox History JavaScript Usage
<script src="https://cdn.pubnub.com/pubnub.min.js"></script>
<script src="mtgox-history.js"></script>
<script>(function(){
// LOAD HOURLY SUMMARY
MTGOX.history.hourly({
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(response) { console.log(JSON.stringify(response)) },
error : function() { console.log("NETWORK ERROR") }
});
// LOAD ALL WITH LIMITER OPTION
MTGOX.history.full({
limit : 500, // SET LIMIT AS HIGH AS NEEDED TO LOAD MORE!
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(messages) { console.log(messages) },
error : function(e) { console.log("NETWORK ERROR") }
});
})();</script>
NOTE: Running MTGOX.history.hourly() method will generate a list of snapshots per hour over the last 24 hours.
NOTE: Running MTGOX.history.full() method will generate maximum resolution detail with a lot of data. You can get a full dump or partial dump as needed; and you should increase the limit parameter in order to grab more data points.
This following JavaScript file will provide you the MTGOX interface.
PubNub Mt.Gox History JavaScript Loader
//
// mtgox-history.js
//
(function(){
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// INITIALIZE PUBNUB
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
var pubnub = PUBNUB.init({
subscribe_key : 'sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe'
});
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// MTGOX HISTORY INTERFACE
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
window.MTGOX = {
history : {
hourly : hourly,
full : full
}
};
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// GET ALL DATA FOREVER (WITH LIMIT OF COURSE)
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/*
MTGOX.history.full({
limit : 1000,
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(messages) { console.log(messages) },
error : function(e) { console.log("NETWORK ERROR") }
});
*/
function full(args) {
var chan = args['channel'] ||'d5f06780-30a8-4a48-a2f8-7ed181b4a13f'
, callback = args['data'] || function(){}
, error = args['error'] || function(){}
, limit = +args['limit'] || 5000
, start = 0
, count = 100
, history = []
, params = {
channel : chan,
count : count,
callback : function(messages) {
var msgs = messages[0];
start = messages[1];
params.start = start;
PUBNUB.each( msgs.reverse(), function(m) {history.push(m)} );
if (history.length >= limit) return callback(history);
if (msgs.length < count) return callback(history);
count = 100;
add_messages();
},
error : function(e) {
callback(history);
error(history);
}
};
add_messages();
function add_messages() { pubnub.history(params) }
}
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// GET 24 HOURS IN HOURLY INCREMENTS
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/*
MTGOX.history.hourly({
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(response) { console.log(response) },
error : function() { console.log('ERROR') }
});
*/
function hourly(setup) {
var limit = 24;
var count = 0;
var chan = setup['channel'] ||'d5f06780-30a8-4a48-a2f8-7ed181b4a13f';
var cb = setup['data'] || function(){};
var eb = setup['error'] || function(){};
var now = new Date();
now.setUTCHours(0);
now.setUTCMinutes(0);
now.setUTCSeconds(0);
now.setUTCMilliseconds(0);
var utc_now = now.getTime();
var vectors = [];
PUBNUB.each( (new Array(limit)).join(',').split(','), function( _, d ) {
var day = utc_now - 3600000 * d;
pubnub.history({
limit : 1,
channel : chan,
start : day * 10000,
error : function() { count++; eb(); },
callback : function(messages) {
// DONE?
if (++count == limit) return cb(vectors);
// ADD TIME SLICES
var res = +(((messages[0][0]||{}).ticker||{}).avg||{}).value;
res && vectors.push([ new Date(day).getUTCHours(), res ]);
// KEEP IT SORTED
vectors.sort(function(a,b){ return a[0] > b[0] && -1 || 1 });
}
})
} );
}
})();
Mt.Gox PubNub Channel Listing for Tickers, Depth and Trades
The following is a list of channels provided by Mt.Gox data feed options you can use in the history channel parameter field.
{
"TICKER.ltcgbp": "0102A446-E4D4-4082-8E83-CC02822F9172",
"TICKER.ltccny": "0290378C-E3D7-4836-8CB1-2BFAE20CC492",
"DEPTH.btchkd": "049F65DC-3AF3-4FFD-85A5-AAC102B2A579",
"DEPTH.btceur": "057BDC6B-9F9C-44E4-BC1A-363E4443CE87",
"TICKER.nmcaud": "08C65460-CBD9-492E-8473-8507DFA66AE6",
"TICKER.btceur": "0BB6DA8B-F6C6-4ECF-8F0D-A544AD948C15",
"DEPTH.btckrw": "0C84BDA7-E613-4B19-AE2A-6D26412C9F70",
"DEPTH.btccny": "0D1ECAD8-E20F-459E-8BED-0BDCF927820F",
"TICKER.btccad": "10720792-084D-45BA-92E3-CF44D9477775",
"DEPTH.btcchf": "113FEC5F-294D-4929-86EB-8CA4C3FD1BED",
"TICKER.ltcnok": "13616AE8-9268-4A43-BDF7-6B8D1AC814A2",
"TICKER.ltcusd": "1366A9F3-92EB-4C6C-9CCC-492A959ECA94",
"TICKER.btcbtc": "13EDFF67-CFA0-4D99-AA76-52BD15D6A058",
"TICKER.ltccad": "18B55737-3F5C-4583-AF63-6EB3951EAD72",
"TICKER.nmccny": "249FDEFD-C6EB-4802-9F54-064BC83908AA",
"DEPTH.btcusd": "24E67E0D-1CAD-4CC0-9E7A-F8523EF460FE",
"TICKER.btcchf": "2644C164-3DB7-4475-8B45-C7042EFE3413",
"DEPTH.btcaud": "296EE352-DD5D-46F3-9BEA-5E39DEDE2005",
"TICKER.btcczk": "2A968B7F-6638-40BA-95E7-7284B3196D52",
"TICKER.btcsgd": "2CB73ED1-07F4-45E0-8918-BCBFDA658912",
"TICKER.nmcjpy": "314E2B7A-A9FA-4249-BC46-B7F662ECBC3A",
"TICKER.btcnmc": "36189B8C-CFFA-40D2-B205-FB71420387AE",
"DEPTH.btcinr": "414FDB18-8F70-471C-A9DF-B3C2740727EA",
"DEPTH.btcsgd": "41E5C243-3D44-4FAD-B690-F39E1DBB86A8",
"TICKER.btcltc": "48B6886F-49C0-4614-B647-BA5369B449A9",
"TICKER.ltceur": "491BC9BB-7CD8-4719-A9E8-16DAD802FFAC",
"TICKER.btcinr": "55E5FEB8-FEA5-416B-88FA-40211541DECA",
"TICKER.ltcjpy": "5AD8E40F-6DF3-489F-9CF1-AF28426A50CF",
"DEPTH.btccad": "5B234CC3-A7C1-47CE-854F-27AEE4CDBDA5",
"TICKER.btcnzd": "5DDD27CA-2466-4D1A-8961-615DEDB68BF1",
"DEPTH.btcgbp": "60C3AF1B-5D40-4D0E-B9FC-CCAB433D2E9C",
"DEPTH.btcnok": "66DA7FB4-6B0C-4A10-9CB7-E2944E046EB5",
"DEPTH.btcthb": "67879668-532F-41F9-8EB0-55E7593A5AB8",
"TICKER.btcsek": "6CAF1244-655B-460F-BEAF-5C56D1F4BEA7",
"TICKER.btcnok": "7532E866-3A03-4514-A4B1-6F86E3A8DC11",
"TICKER.btcgbp": "7B842B7D-D1F9-46FA-A49C-C12F1AD5A533",
"TRADE.LAG": "85174711-BE64-4DE1-B783-0628995D7914",
"DEPTH.btcsek": "8F1FEFAA-7C55-4420-ADA0-4DE15C1C38F3",
"DEPTH.btcdkk": "9219ABB0-B50C-4007-B4D2-51D1711AB19C",
"DEPTH.btcjpy": "94483E07-D797-4DD4-BC72-DC98F1FD39E3",
"TICKER.nmcusd": "9AAEFD15-D101-49F3-A2FD-6B63B85B6BED",
"TICKER.ltcaud": "A046600A-A06C-4EBF-9FFB-BDC8157227E8",
"TICKER.btcjpy": "A39AE532-6A3C-4835-AF8C-DDA54CB4874E",
"DEPTH.btcczk": "A7A970CF-4F6C-4D85-A74E-AC0979049B87",
"TICKER.ltcdkk": "B10A706E-E8C7-4EA8-9148-669F86930B36",
"TICKER.btcpln": "B4A02CB3-2E2D-4A88-AEEA-3C66CB604D01",
"TEST": "BAD99F24-FA8B-4938-BFDF-0C1831FC6665",
"TICKER.btcrub": "BD04F720-3C70-4DCE-AE71-2422AB862C65",
"TICKER.nmcgbp": "BF5126BA-5187-456F-8AE6-963678D0607F",
"TICKER.btckrw": "BF85048D-4DB9-4DBE-9CA3-5B83A1A4186E",
"TICKER.btccny": "C251EC35-56F9-40AB-A4F6-13325C349DE4",
"DEPTH.btcnzd": "CEDF8730-BCE6-4278-B6FE-9BEE42930E95",
"TICKER.btchkd": "D3AE78DD-01DD-4074-88A7-B8AA03CD28DD",
"TICKER.btcthb": "D58E3B69-9560-4B9E-8C58-B5C0F3FDA5E1",
"TICKER.btcusd": "D5F06780-30A8-4A48-A2F8-7ED181B4A13F",
"DEPTH.btcrub": "D6412CA0-B686-464C-891A-D1BA3943F3C6",
"TICKER.nmceur": "D8512D04-F262-4A14-82F2-8E5C96C15E68",
"TRADE.btc": "DBF1DEE9-4F2E-4A08-8CB7-748919A71B21",
"TICKER.nmccad": "DC28033E-7506-484C-905D-1C811A613323",
"DEPTH.btcpln": "E4FF055A-F8BF-407E-AF76-676CAD319A21",
"TICKER.btcdkk": "E5CE0604-574A-4059-9493-80AF46C776B3",
"TICKER.btcaud": "EB6AAA11-99D0-4F64-9E8C-1140872A423D"
}
See https://help.pubnub.com/entries/24113341-How-do-I-Page-Through-Stored-Messages-
Contact PubNub support (help#...) if further assistance is needed
The below Java class can be used to easily retrieve and process long time ranges of history messages in an advanced for loop of this form:
PubnubHistoryExcerpt history = new PubnubHistoryExcerpt(pubnub, channel, start, end);
for (Object message : history) {
// do something with the message object
}
The messages are retrieved on the fly, so no memory problem occurs.
Below is the complete code. You can find a fully runnable usage example in the main() method inside of the class.
I haven't yet tested this class extensively. Enhancements are welcome.
/*
* PubnubHistoryExcerpt.java
*
* This file is distributed under the FreeBSD License:
*
* Copyright (c) 2014, Daniel S. (http://stackoverflow.com/users/1838726/daniel-s)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are those
* of the authors and should not be interpreted as representing official policies,
* either expressed or implied, of the FreeBSD Project.
*/
import java.math.BigDecimal;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.NoSuchElementException;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import com.pubnub.api.Callback;
import com.pubnub.api.Pubnub;
import com.pubnub.api.PubnubError;
/**
* You can use this class to iterate over historical PubNub messages. The messages are retrieved transparently while you
* iterate over the history excerpt. This class and the returned iterators are thread-safe.
*
* See {#link #main(String[])} for a usage example.
*/
public class PubnubHistoryExcerpt implements Iterable<Object> {
/**
* This main method contains a usage example for this class. It downloads last 3 hour's messages of the MtGox BTC/USD ticker
* channel from PubNub and outputs the timestamp and USD value which are found in the messages.
*/
public static void main(String[] args) throws JSONException {
String PUBNUB_SUBSCRIBE_KEY_MTGOX = "sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe";
String PUBNUB_CHANNEL_MTGOX_TICKER_BTCUSD = "d5f06780-30a8-4a48-a2f8-7ed181b4a13f";
Pubnub pubnub = new Pubnub(null, PUBNUB_SUBSCRIBE_KEY_MTGOX);
long ONE_HOUR_IN_MILLIS = 60 * 60 * 1000;
long end = System.currentTimeMillis();
long start = end - 3 * ONE_HOUR_IN_MILLIS;
// convert from milliseconds as time unit (10^-3 seconds) to
// pubnub's better-than-microsecond precision time units (10^-7 seconds)
start *= 10000;
end *= 10000;
PubnubHistoryExcerpt history = new PubnubHistoryExcerpt(pubnub, PUBNUB_CHANNEL_MTGOX_TICKER_BTCUSD, start, end);
DefaultDateFormat dateFormat = DefaultDateFormat.create();
for (Object message : history) {
JSONObject messageJson = (JSONObject) message;
JSONObject ticker = messageJson.getJSONObject("ticker");
long instant = ticker.getLong("now");
BigDecimal value = new BigDecimal(ticker.getJSONObject("last_local").getString("value"));
instant /= 1000; // convert from microseconds to milliseconds
System.out.println(dateFormat.format(instant) + ": " + value);
}
System.exit(0);
}
/**
* This is the maximum number of messages to fetch in one batch. If you fetch many messages, higher numbers improve
* performance. Setting this to a value higher than 100 doesn't have an effect, because Pubnub currently doesn't
* support fetching more than this many messages at once.
*/
private static final int BATCH_SIZE = 100;
private final Pubnub pubnub;
private final String channel;
private final long start;
private final long end;
/**
* Constructs a new excerpt over which you can iterate. Insances represent an excerpt. No retrieval operations are
* started unless you call iterator().next() for the first time.
*
* #param pubnub
* The Pubnub connection to use for retrieving messages.
* #param channel
* The channel for which to retrieve historical messages.
* #param start
* The beginning of the time interval for which to retrieve messages, in pubnub's time units (10^-7
* seconds, so milliseconds * 10000) since 1970-01-01 00:00:00).
* #param end
* The end of the time interval for which to retrieve messages, in pubnub's time units (10^-7 seconds, so
* milliseconds * 10000) since 1970-01-01 00:00:00).
*/
private PubnubHistoryExcerpt(Pubnub pubnub, String channel, long start, long end) {
this.pubnub = pubnub;
this.channel = channel;
this.start = start;
this.end = end;
}
public Iterator<Object> iterator() {
return new Iter();
}
private class Iter implements Iterator<Object> {
/**
* This list is used as a fifo buffer for messages retrieves through this iterator. It also acts as the main
* synchronization lock for synchronizing access between threads accessing this class as an iterator and threads
* calling back from the Pubnub API.
*/
private LinkedList<Object> buffer = new LinkedList<Object>();
/**
* This field stores the end of the time range of the previous batch retrieval, in Pubnub time units (10th of a
* microsecond, so milliseconds*10000). For the following batch retrieval, this is used as the start time for
* retrieving the following messages.
*/
private long prevBatchTimeRangeEnd = PubnubHistoryExcerpt.this.start;
/**
* Retrieval of messages is handled asynchronously. That means that exceptions which are thrown during retrieval
* can't automatically be propagated through to the code which invokes <code>next()</code> or
* <code>hasNext()</code> . Therefor, such an exception is stored temporarily in this field and then re-thrown
* from within <code>next()</code> or <code>hasNext()</code>.
*/
private Exception caughtDuringRetrieval = null;
/**
* This object is used to wait on and to notify about updates of the buffer.
*/
private Object notifier = new Object();
/**
* Because of spurious wakeups that can happen during wait(), this field is necessary to tell the waiting thread
* if retrieval is still running.
*/
private boolean retrieving = false;
/**
* The callback object to use for retrieving messages. This is stored in a field here for re-use. This is a
* compromise between performance and low memory footprint, slightly in favor of performance.
*/
private InternalCallback internalCallback = new InternalCallback();
private void retrieveNextBatch() {
synchronized (notifier) {
this.retrieving = true;
// String startStr = DefaultDateFormat.create().format(prevBatchTimeRangeEnd / 10000);
// String endStr = DefaultDateFormat.create().format(end / 10000);
// System.out.println("fetching from " + startStr + " till " + endStr);
if (Iter.this.prevBatchTimeRangeEnd < PubnubHistoryExcerpt.this.end) {
PubnubHistoryExcerpt.this.pubnub.history( //
PubnubHistoryExcerpt.this.channel, //
Iter.this.prevBatchTimeRangeEnd, //
PubnubHistoryExcerpt.this.end, //
BATCH_SIZE, //
false, //
Iter.this.internalCallback //
);
waitUntilNextBatchRetrievalFinished();
}
}
}
private void waitUntilNextBatchRetrievalFinished() {
while (this.retrieving) {
try {
this.notifier.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private class InternalCallback extends Callback {
#Override
public void successCallback(String channel, Object message) {
synchronized (Iter.this.notifier) {
try {
processSuccessCallback(channel, message);
} catch (Exception e) {
Iter.this.caughtDuringRetrieval = e;
} finally {
Iter.this.retrieving = false;
Iter.this.notifier.notifyAll();
}
}
}
#Override
public void errorCallback(String channel, PubnubError error) {
Iter.this.caughtDuringRetrieval = new Exception("" + //
error.getClass().getName() + ": " + //
error.getErrorString() + //
" (code=" + error.errorCode + "; extendedCode=" + error.errorCodeExtended + ")");
Iter.this.caughtDuringRetrieval.fillInStackTrace();
}
}
private void processSuccessCallback(String channel, Object message) throws JSONException {
if (message == null)
throw new NullPointerException("retrieved message is null");
if (!(message instanceof JSONArray))
throw new RuntimeException("retrieved message is not a " + JSONArray.class.getName());
JSONArray historyMessage = (JSONArray) message;
// System.out.println(historyMessage.toString(2));
JSONArray messageList = extractMessageList(historyMessage);
long batchTimeRangeEnd = extractBatchTimeRangeEnd(historyMessage);
if (batchTimeRangeEnd > 0)
Iter.this.prevBatchTimeRangeEnd = batchTimeRangeEnd;
else
Iter.this.prevBatchTimeRangeEnd = end;
processMessageList(messageList);
}
private void processMessageList(JSONArray messageList) {
int i = 0;
for (; i < messageList.length(); i++) {
JSONObject message;
try {
message = messageList.getJSONObject(i);
} catch (JSONException e) {
String str;
try {
str = messageList.toString(2);
} catch (JSONException secondaryE) {
str = "(couldn't convert messageList to String because of " + secondaryE.toString() + ")";
}
throw new RuntimeException("couldn't extract message at index " + i + " from messageList (messageList:\n" + str
+ "\n(end of messageList)\n)", e);
}
Iter.this.buffer.add(message);
}
}
private long extractBatchTimeRangeEnd(JSONArray historyMessage) {
long batchTimeRangeEnd;
try {
batchTimeRangeEnd = historyMessage.getLong(2);
} catch (JSONException e) {
String str = safeConvertHistoryMessageToString(historyMessage);
throw new RuntimeException("could not extract element 2 (batchTimeRangeEnd) of retrieved historyMessage (historyMessage:\n" + str
+ "\n(end of historyMessage)\n)", e);
}
return batchTimeRangeEnd;
}
private String safeConvertHistoryMessageToString(JSONArray historyMessage) {
String str;
try {
str = historyMessage.toString(2);
} catch (JSONException secondaryE) {
str = "(couldn't convert historyMessage to String because of " + secondaryE.toString() + ")";
}
return str;
}
private JSONArray extractMessageList(JSONArray historyMessage) {
JSONArray messageArJson;
try {
messageArJson = historyMessage.getJSONArray(0);
} catch (JSONException e) {
String str = safeConvertHistoryMessageToString(historyMessage);
throw new RuntimeException("could not extract element 0 (messageList) of retrieved historyMessage (historyMessage:\n" + str
+ "\n(end of historyMessage)\n)", e);
}
return messageArJson;
}
public boolean hasNext() {
synchronized (Iter.this.buffer) {
ensureNotInExceptionState();
if (Iter.this.buffer.isEmpty())
retrieveNextBatch();
return !Iter.this.buffer.isEmpty();
}
}
public Object next() {
synchronized (Iter.this.buffer) {
if (!hasNext()) {
throw new NoSuchElementException("there are no more elements in this iterator");
}
Object result = Iter.this.buffer.removeFirst();
return result;
}
}
private void ensureNotInExceptionState() {
if (caughtDuringRetrieval != null) {
throw new RuntimeException("an exception was caught already by a previous attempt to access this iterator", caughtDuringRetrieval);
}
}
public void remove() {
throw new UnsupportedOperationException(getClass().getName() + " doesn't support remove()");
}
}
}

How do I debug a form which does not send an email?

I have a form build from the kohana framework which should send an email.
when I press the "send" button, everything seems to work fine...no error messages...but no email turns up!
If I can´t see anything in firebug..where can I look?
the application logs state
"error: Missing i18n entry contact.captcha.valid for language en_US"
but I don´t know how to get to the bottom of the problem..any help welcome..
yours,
Rob
Ill try and find out which version im using.....the application is the latest version of Ushahidi (2.2.1) www.ushahidi.com
<?php defined('SYSPATH') OR die('No direct access allowed.');
/**
* Captcha library.
*
* $Id: Captcha.php 3917 2009-01-21 03:06:22Z zombor $
*
* #package Captcha
* #author Kohana Team
* #copyright (c) 2007-2008 Kohana Team
* #license http://kohanaphp.com/license.html
*/
class Captcha_Core {
// Captcha singleton
protected static $instance;
// Style-dependent Captcha driver
protected $driver;
// Config values
public static $config = array
(
'style' => 'basic',
'width' => 150,
'height' => 50,
'complexity' => 4,
'background' => '',
'fontpath' => '',
'fonts' => array(),
'promote' => FALSE,
);
/**
* Singleton instance of Captcha.
*
* #return object
*/
public static function instance()
{
// Create the instance if it does not exist
empty(self::$instance) and new Captcha;
return self::$instance;
}
/**
* Constructs and returns a new Captcha object.
*
* #param string config group name
* #return object
*/
public static function factory($group = NULL)
{
return new Captcha($group);
}
/**
* Constructs a new Captcha object.
*
* #throws Kohana_Exception
* #param string config group name
* #return void
*/
public function __construct($group = NULL)
{
// Create a singleton instance once
empty(self::$instance) and self::$instance = $this;
// No config group name given
if ( ! is_string($group))
{
$group = 'default';
}
// Load and validate config group
if ( ! is_array($config = Kohana::config('captcha.'.$group)))
throw new Kohana_Exception('captcha.undefined_group', $group);
// All captcha config groups inherit default config group
if ($group !== 'default')
{
// Load and validate default config group
if ( ! is_array($default = Kohana::config('captcha.default')))
throw new Kohana_Exception('captcha.undefined_group', 'default');
// Merge config group with default config group
$config += $default;
}
// Assign config values to the object
foreach ($config as $key => $value)
{
if (array_key_exists($key, self::$config))
{
self::$config[$key] = $value;
}
}
// Store the config group name as well, so the drivers can access it
self::$config['group'] = $group;
// If using a background image, check if it exists
if ( ! empty($config['background']))
{
self::$config['background'] = str_replace('\\', '/', realpath($config['background']));
if ( ! is_file(self::$config['background']))
throw new Kohana_Exception('captcha.file_not_found', self::$config['background']);
}
// If using any fonts, check if they exist
if ( ! empty($config['fonts']))
{
self::$config['fontpath'] = str_replace('\\', '/', realpath($config['fontpath'])).'/';
foreach ($config['fonts'] as $font)
{
if ( ! is_file(self::$config['fontpath'].$font))
throw new Kohana_Exception('captcha.file_not_found', self::$config['fontpath'].$font);
}
}
// Set driver name
$driver = 'Captcha_'.ucfirst($config['style']).'_Driver';
// Load the driver
if ( ! Kohana::auto_load($driver))
throw new Kohana_Exception('core.driver_not_found', $config['style'], get_class($this));
// Initialize the driver
$this->driver = new $driver;
// Validate the driver
if ( ! ($this->driver instanceof Captcha_Driver))
throw new Kohana_Exception('core.driver_implements', $config['style'], get_class($this), 'Captcha_Driver');
Kohana::log('debug', 'Captcha Library initialized');
}
/**
* Validates a Captcha response and updates response counter.
*
* #param string captcha response
* #return boolean
*/
public static function valid($response)
{
// Maximum one count per page load
static $counted;
// User has been promoted, always TRUE and don't count anymore
if (self::instance()->promoted())
return TRUE;
// Challenge result
$result = (bool) self::instance()->driver->valid($response);
// Increment response counter
if ($counted !== TRUE)
{
$counted = TRUE;
// Valid response
if ($result === TRUE)
{
self::instance()->valid_count(Session::instance()->get('captcha_valid_count') + 1);
}
// Invalid response
else
{
self::instance()->invalid_count(Session::instance()->get('captcha_invalid_count') + 1);
}
}
return $result;
}
/**
* Gets or sets the number of valid Captcha responses for this session.
*
* #param integer new counter value
* #param boolean trigger invalid counter (for internal use only)
* #return integer counter value
*/
public function valid_count($new_count = NULL, $invalid = FALSE)
{
// Pick the right session to use
$session = ($invalid === TRUE) ? 'captcha_invalid_count' : 'captcha_valid_count';
// Update counter
if ($new_count !== NULL)
{
$new_count = (int) $new_count;
// Reset counter = delete session
if ($new_count < 1)
{
Session::instance()->delete($session);
}
// Set counter to new value
else
{
Session::instance()->set($session, (int) $new_count);
}
// Return new count
return (int) $new_count;
}
// Return current count
return (int) Session::instance()->get($session);
}
/**
* Gets or sets the number of invalid Captcha responses for this session.
*
* #param integer new counter value
* #return integer counter value
*/
public function invalid_count($new_count = NULL)
{
return $this->valid_count($new_count, TRUE);
}
/**
* Resets the Captcha response counters and removes the count sessions.
*
* #return void
*/
public function reset_count()
{
$this->valid_count(0);
$this->valid_count(0, TRUE);
}
/**
* Checks whether user has been promoted after having given enough valid responses.
*
* #param integer valid response count threshold
* #return boolean
*/
public function promoted($threshold = NULL)
{
// Promotion has been disabled
if (self::$config['promote'] === FALSE)
return FALSE;
// Use the config threshold
if ($threshold === NULL)
{
$threshold = self::$config['promote'];
}
// Compare the valid response count to the threshold
return ($this->valid_count() >= $threshold);
}
/**
* Returns or outputs the Captcha challenge.
*
* #param boolean TRUE to output html, e.g. <img src="#" />
* #return mixed html string or void
*/
public function render($html = TRUE)
{
return $this->driver->render($html);
}
/**
* Magically outputs the Captcha challenge.
*
* #return mixed
*/
public function __toString()
{
return $this->render();
}
} // End Captcha Class
From the little details provided I think your site uses the I18n library from Kohana for internationalization of the site. The other option is your site use Kohana messages for showing form errors.
The Validation class uses the __() function internally when generation validation errors. I think you don't have a message for key valid specified in APPPATH/messages/contact/captcha.php.
You should try to investigate more on how the form is processed and whether some validation errors are being generated. There might be an error in the captcha validation and despite it seem there is no error, there might be and just the error message could not be shown.
class Contact_Controller extends Main_Controller
{
function __construct()
{
parent::__construct();
}
public function index()
{
$this->template->header->this_page = 'contact';
$this->template->content = new View('contact');
$this->template->header->page_title .= Kohana::lang('ui_main.contact').Kohana::config('settings.title_delimiter');
// Setup and initialize form field names
$form = array (
'contact_name' => '',
'contact_email' => '',
'contact_phone' => '',
'contact_subject' => '',
'contact_message' => '',
'captcha' => ''
);
// Copy the form as errors, so the errors will be stored with keys
// corresponding to the form field names
$captcha = Captcha::factory();
$errors = $form;
$form_error = FALSE;
$form_sent = FALSE;
// Check, has the form been submitted, if so, setup validation
if ($_POST)
{
// Instantiate Validation, use $post, so we don't overwrite $_POST fields with our own things
$post = Validation::factory($_POST);
// Add some filters
$post->pre_filter('trim', TRUE);
// Add some rules, the input field, followed by a list of checks, carried out in order
$post->add_rules('contact_name', 'required', 'length[3,100]');
$post->add_rules('contact_email', 'required','email', 'length[4,100]');
$post->add_rules('contact_subject', 'required', 'length[3,100]');
$post->add_rules('contact_message', 'required');
$post->add_rules('captcha', 'required', 'Captcha::valid');
// Test to see if things passed the rule checks
if ($post->validate())
{
// Yes! everything is valid - Send email
$site_email = Kohana::config('settings.site_email');
$message = Kohana::lang('ui_admin.sender').": " . $post->contact_name . "\n";
$message .= Kohana::lang('ui_admin.email').": " . $post->contact_email . "\n";
$message .= Kohana::lang('ui_admin.phone').": " . $post->contact_phone . "\n\n";
$message .= Kohana::lang('ui_admin.message').": \n" . $post->contact_message . "\n\n\n";
$message .= "~~~~~~~~~~~~~~~~~~~~~~\n";
$message .= Kohana::lang('ui_admin.sent_from_website'). url::base();
// Send Admin Message
email::send( $site_email, $post->contact_email, $post->contact_subject, $message, FALSE );
$form_sent = TRUE;
}
// No! We have validation errors, we need to show the form again, with the errors
else
{
// repopulate the form fields
$form = arr::overwrite($form, $post->as_array());
// populate the error fields, if any
$errors = arr::overwrite($errors, $post->errors('contact'));
$form_error = TRUE;
}
}
$this->template->content->form = $form;
$this->template->content->errors = $errors;
$this->template->content->form_error = $form_error;
$this->template->content->form_sent = $form_sent;
$this->template->content->captcha = $captcha;
// Rebuild Header Block
$this->template->header->header_block = $this->themes->header_block();
$this->template->footer->footer_block = $this->themes->footer_block();
}
}
My problem has been solved!
My Email server was not set up properly. I resolved the postfix issue and now the comments form is working as it should,
thanks to all who answered,
Rob

Resources