Programmatically create a new Person having an Internal Login - sas-metadata

I am trying to programmatically create Users with Internal Accounts as part of a testing system. The following code can not create an InternalLogin because there is not password hash set at object creation time.
Can a person + internal account be created using metadata_* functions ?
data _null_;
length uri_Person uri_PW uri_IL $256;
call missing (of uri:);
rc = metadata_getnobj ("Person?#Name='testbot02'", 1, uri_Person); msg=sysmsg();
put 'NOTE: Get Person, ' rc= uri_Person= / msg;
if rc = -4 then do;
rc = metadata_newobj ('Person', uri_Person, 'testbot02'); msg=sysmsg();
put 'NOTE: New Person, ' rc= uri_Person= / msg;
end;
rc = metadata_setattr (uri_Person, 'DisplayName', 'Test Bot #2'); msg=sysmsg();
put 'NOTE: SetAttr, ' rc= / msg;
rc = metadata_newobj ('InternalLogin', uri_IL, 'autobot - IL', 'Foundation', uri_Person, 'InternalLoginInfo'); msg=sysmsg();
put 'NOTE: New InternalLogin, ' rc= / msg;
run;
Logs
NOTE: Get Person, rc=-4 uri_Person=
NOTE: New Person, rc=0 uri_Person=OMSOBJ:Person\A5OJU4RB.AP0000SX
NOTE: SetAttr, rc=0
NOTE: New InternalLogin, rc=-2
ERROR: AddMetadata of InternalLogin is missing required property PasswordHash.
NOTE: DATA statement used (Total process time):
real time 0.02 seconds
cpu time 0.01 seconds
The management console and metabrowse were used to find what objects might need to be created.
Metabrowse

I ended up using Java to create new users.
A ISecurity_1_1 instance from MakeISecurityConnection() on the metadata connection was used to SetInternalPassword
Java ended up being a lot more clear coding wise.
package sandbox.sas.metadata;
import com.sas.iom.SASIOMDefs.GenericError;
import com.sas.metadata.remote.*;
import com.sas.meta.SASOMI.*;
import java.rmi.RemoteException;
import java.util.List;
/**
*
* #author Richard
*/
public class Main {
public static final String TESTGROUPNAME = "Automated Test Users";
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
String metaserver="################";
String metaport="8561";
String omrUser="sasadm#saspw";
String omrPass="##########";
try
{
MdFactoryImpl metadata = new MdFactoryImpl(false);
metadata.makeOMRConnection(metaserver,metaport,omrUser,omrPass);
MdOMIUtil util = metadata.getOMIUtil();
ISecurity_1_1 security = metadata.getConnection().MakeISecurityConnection();
MdObjectStore workunit = metadata.createObjectStore();
String foundationID = util.getFoundationReposID();
String foundationShortID = foundationID.substring(foundationID.indexOf(".") + 1);
// Create group for test bot accounts
List identityGroups = util.getMetadataObjects(foundationID, MetadataObjects.IDENTITYGROUP, MdOMIUtil.OMI_XMLSELECT,
String.format("<XMLSelect search=\"IdentityGroup[#Name='%s']\"/>", TESTGROUPNAME));
IdentityGroup identityGroup;
if (identityGroups.isEmpty()) {
identityGroup = (IdentityGroup) metadata.createComplexMetadataObject (workunit, TESTGROUPNAME, MetadataObjects.IDENTITYGROUP, foundationShortID);
identityGroup.setDisplayName(TESTGROUPNAME);
identityGroup.setDesc("Group for Automated Test Users performing concurrent Stored Process execution");
identityGroup.updateMetadataAll();
}
identityGroups = util.getMetadataObjectsSubset(workunit, foundationID, MetadataObjects.IDENTITYGROUP, MdOMIUtil.OMI_XMLSELECT,
String.format("<XMLSelect search=\"IdentityGroup[#Name='%s']\"/>", TESTGROUPNAME));
identityGroup = (IdentityGroup) identityGroups.get(0);
// Create (or Update) test bot accounts
for (int index = 1; index <= 25; index++)
{
String token = String.format("%02d", index);
String personName = String.format("testbot%s", token);
String password = String.format("testbot%s%s", token, token); * simple dynamically generated password for user (for testing scripts only);
String criteria = String.format("Person[#Name='%s']", personName);
String xmlSelect = String.format("<XMLSelect search=\"%s\"/>", criteria);
List persons = util.getMetadataObjectsSubset(workunit, foundationID, MetadataObjects.PERSON, MdOMIUtil.OMI_XMLSELECT | MdOMIUtil.OMI_GET_METADATA | MdOMIUtil.OMI_ALL_SIMPLE, xmlSelect);
Person person;
if (persons.size() == 1)
{
person = (Person) persons.get(0);
System.out.println(String.format("Have %s %s", person.getName(), person.getDisplayName()));
}
else
{
person = (Person) metadata.createComplexMetadataObject (workunit, personName, MetadataObjects.PERSON, foundationShortID);
person.setDisplayName(String.format("Test Bot #%s", token));
person.setDesc("Internal account for testing purposes");
System.out.println(String.format("Make %s, %s (%s)", person.getName(), person.getDisplayName(), person.getDesc()));
person.updateMetadataAll();
}
security.SetInternalPassword(personName, password);
AssociationList personGroups = person.getIdentityGroups();
personGroups.add(identityGroup);
person.setIdentityGroups(personGroups);
person.updateMetadataAll();
}
workunit.dispose();
metadata.closeOMRConnection();
metadata.dispose();
}
catch (GenericError | MdException | RemoteException e)
{
System.out.println(e.getLocalizedMessage());
System.exit(1);
}
}
}

Related

Payment field not working

I have used SOAP API to create Embedded signing. I am using template (CreateEnvelopeFromTemplates) to create envelope. I have added a payment field on template which works fine if I send envelope request from docusign website. I get a popup to enter payment information. BUT when I am sending the same template from API then Payment functionality is not working. Payment field is appearing as number.
screenshot
Here is my code
public class DS_Recipe_Signer_View_Controller {
// Embedded signing of an envelope
// Copyright (c) 2016 DocuSign, Inc.
// LICENSE: The MIT License, see https://opensource.org/licenses/MIT
// SETTINGS
// Private static string integration_key = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx';
// Private static string account_id = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx';
Private static string integration_key = 'XXXXXXXXXX-98333677e8bd';
Private static string account_id = 'XXXXXXXXXXX-5f66c17b7f9f';
// NOTE: You MUST use the long form of the account id. It's has 32 digits
// with 4 dashes (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).
// This version of the account id is shown in the APIs and Connects section
// of your DocuSign Administration tool
Public string signer_email {get;set;} // Required
Public string signer_name {get;set;} // Required
Public string email_message {get;set;} // Required
Public string signer_user_id {get;set;} // Required for embedded signing
Public string signer_return_url {get;set;} // Required. Where DS redirects to after the signing ceremony
Public string output {get;set;}
Public string envelope_id {get;set;}
Public string signer_view_url {get;set;} // Redirect to this url
Public string error_code {get;set;} // Null means no error
Public string error_message {get;set;}
// Using Legacy authentication via an SFDC Named Credential
Private static string ds_server = 'callout:DocuSign_Legacy_Demo/api/3.0/dsapi.asmx';
// If you choose to not use a named credential:
//Private static string ds_server = 'https://requestb.in/119qunn1';
//Private static string ds_server ='https://demo.docusign.net/api/3.0/dsapi.asmx';
Private static string trace_value = 'SFDC_002_SOAP_embedded_signing'; // Used for tracing API calls
Private static string trace_key = 'X-ray';
Private DocuSignTK.APIServiceSoap api_sender = new DocuSignTK.APIServiceSoap();
Public DS_Recipe_Signer_View_Controller(){}
Public void send(){
configure_sender();
send_envelope();
embedded_signing();
if (no_error()) {
output = '<p>The envelope was sent, Envelope ID: ' + envelope_id + '</p>';
output += '<p></p><p>Signer: ' + signer_name + ' <' + signer_email + '></p>';
output += '<p><b>To sign the envelope, redirect the user to the <a href = "' +
signer_view_url + '" target="_blank">DocuSign Signing Ceremony</a></b></p>';
output += '<p>The redirect address is ' + signer_view_url + '</p>';
output += '<p><b>Note:</b> the Signing Ceremony url can only be used for a couple of minutes after ' +
'it has been created. Do NOT store the url for later use. Instead, ' +
'generate the URL immediately before you redirect the user\'s browser.</p>';
output += '<p>After the signer has completed the signing ceremony, his ' +
'browser will be redirected back to your app with some query fields added. Example: </p>' +
'<p>http://www.foo.com/?event=signing_complete</p>';
} else {
output = '<h3>Problem</h3><p>' + error_message + '</p>';
}
}
Private void configure_sender(){
api_sender.endpoint_x = ds_server;
api_sender.inputHttpHeaders_x = new Map<String, String>();
String auth = '<DocuSignCredentials><Username>{!$Credential.Username}</Username>'
+ '<Password>{!$Credential.Password}</Password>'
+ '<IntegratorKey>' + integration_key + '</IntegratorKey></DocuSignCredentials>';
api_sender.inputHttpHeaders_x.put('X-DocuSign-Authentication', auth);
api_sender.inputHttpHeaders_x.put(trace_key, trace_value);
}
Private void embedded_signing() {
// Obtains the embedded Signing Ceremony URL for an envelope's recipient (the signer).
// To use embedded signing:
// 1. The signer must have been added to the envelope as a "captive signer"
// 2. You need the following values:
// 1. EnvelopeID
// 2. Signer's Email that was provided when the signer was added to the envelope.
// 3. Signer's name (UserName field)
// 4. The Signer's User ID (client id) within your app. Must uniquely identify the signer.
// 3. You also need to create an "Assertion" object where you provide information on how
// your app authenticated the signer. This information is stored by DocuSign so you can
// later use the data in case of a dispute.
// Incoming variables used:
// envelope_id, signer_user_id, signer_email, signer_name
// Maintaining state: when DocuSign redirects back to your app after the signing ceremony
// ended, how does your app know what is going on? You can include additional query parameters
// in the signer_return_url that you supply. Eg the recipient ID, envelope ID, etc.
// You can include your app's sessionID. You can use the cookie system to store either
// specific information or your stack's session id for your app.
// Step 1. Create the assertion
DocuSignTK.RequestRecipientTokenAuthenticationAssertion assertion =
new DocuSignTK.RequestRecipientTokenAuthenticationAssertion();
assertion.AssertionID = '1'; // A unique identifier of the authentication
// event executed by your application.
assertion.AuthenticationInstant = Datetime.now(); // The date/time that the end-user was authenticated.
assertion.AuthenticationMethod = 'Password'; // How did your app authenticate the signer?
// Options: Password, Email, PaperDocuments, HTTPBasicAuth, SSLMutualAuth, X509Certificate, Kerberos,
// SingleSignOn_CASiteminder, SingleSignOn_InfoCard, SingleSignOn_MicrosoftActiveDirectory, SingleSignOn_Passport,
// SingleSignOn_SAML, SingleSignOn_Other, Smartcard, RSASecureID, Biometric, None, KnowledgeBasedAuth
assertion.SecurityDomain = 'DS_Recipe_Signer_View_Controller'; // The "domain" (app, sso system, etc)
// to which the user authenticated
// Step 2. Create the redirect URLs for the different outcomes of the Signing Ceremony
DocuSignTK.RequestRecipientTokenClientURLs urls = new DocuSignTK.RequestRecipientTokenClientURLs();
String return_url_base = signer_return_url;
// The supplied url may already include one or more query parameters. In that case, we're appending
// one more query parameters. Otherwiser, we're adding the first set of query parameters.
// Look for a ? to see if the url already includes query parameters
If (return_url_base.contains('?')) {
return_url_base += '&event=';
} Else {
return_url_base += '?event=';
}
urls.OnSigningComplete = return_url_base + 'signing_complete';
urls.OnViewingComplete = return_url_base + 'viewing_complete';
urls.OnCancel = return_url_base + 'cancel';
urls.OnDecline = return_url_base + 'decline';
urls.OnSessionTimeout = return_url_base + 'session_timeout';
urls.OnTTLExpired = return_url_base + 'ttl_expired';
urls.OnException = return_url_base + 'exception';
urls.OnAccessCodeFailed = return_url_base + 'failed_access_code';
urls.OnIdCheckFailed = return_url_base + 'failed_id_check';
urls.OnFaxPending = return_url_base + 'fax_pending';
// Step 3. Make the call
try {
signer_view_url = api_sender.RequestRecipientToken(
envelope_id, signer_user_id, signer_name, signer_email, assertion, urls);
System.debug('Received signer_view_url: = ' + signer_view_url);
} catch ( CalloutException e) {
System.debug('Exception - ' + e );
error_code = 'Problem: ' + e;
error_message = error_code;
}
}
Private void send_envelope() {
// Sends an envelope. The first signer is "captive," so he can sign embedded
// Check input
if (String.isBlank(signer_email) || String.isBlank(signer_name) || !signer_email.contains('#')) {
error_message = 'Please fill in the email and name fields';
error_code = 'INPUT_PROBLEM';
return;
}
// Check configuration
if (integration_key == 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ||
account_id == 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx') {
error_message = 'Please configure the Apex class DS_Recipe_Send_Env_Email_Controller with your integration key and account id.';
error_code = 'CONFIGURATION_PROBLEM';
return;
}
DocuSignTK.Recipient recipient = new DocuSignTK.Recipient();
recipient.Email = signer_email;
recipient.UserName = signer_name;
recipient.ID = 1;
recipient.Type_x = 'Signer';
recipient.RoutingOrder = 1;
recipient.RoleName = 'Signer';
// We want this signer to be "captive" so we can use embedded signing with him
recipient.CaptiveInfo = new DocuSignTK.RecipientCaptiveInfo();
recipient.CaptiveInfo.ClientUserID = signer_user_id; // Must uniquely identify the
DocuSignTK.ArrayOfRecipient1 recipients = new DocuSignTK.ArrayOfRecipient1();
recipients.Recipient = new DocuSignTK.Recipient[1];
recipients.Recipient[0] = recipient;
// Create the template reference from a server-side template ID
DocuSignTK.TemplateReference templateReference = new DocuSignTK.TemplateReference();
templateReference.Template = '37041580-9d59-4b41-a7c4-7c9bfa3b26c8';//'d0d80082-612b-4a04-b2a1-0672eb720491';
templateReference.TemplateLocation = 'Server';
//templateReference.Sequence = 1;
DocuSignTK.ArrayOfTemplateReferenceRoleAssignment Roles = new DocuSignTK.ArrayOfTemplateReferenceRoleAssignment();
Roles.RoleAssignment = new DocuSignTK.TemplateReferenceRoleAssignment[1];
DocuSignTK.TemplateReferenceRoleAssignment role = new DocuSignTK.TemplateReferenceRoleAssignment();
role.RoleName = 'Signer';
role.RecipientID = 1;
Roles.RoleAssignment[0] = role;
templateReference.RoleAssignments = Roles;
// Construct the envelope information
DocuSignTK.EnvelopeInformation envelopeInfo = new DocuSignTK.EnvelopeInformation();
envelopeInfo.AccountId = account_Id;
envelopeInfo.Subject = 'Subject';
envelopeInfo.EmailBlurb = 'Email content';
DocuSignTK.TemplateReferenceFieldDataDataValue fd2 = new
DocuSignTK.TemplateReferenceFieldDataDataValue();
fd2.TabLabel = 'name';
fd2.Value = 'recipient.UserName';
DocuSignTK.TemplateReferenceFieldDataDataValue fd3 = new
DocuSignTK.TemplateReferenceFieldDataDataValue();
fd3.TabLabel = 'company';
fd3.Value = 'company';
templateReference.FieldData = new DocuSignTK.TemplateReferenceFieldData();
templateReference.FieldData.DataValues = new
DocuSignTK.ArrayOfTemplateReferenceFieldDataDataValue();
templateReference.FieldData.DataValues.DataValue = new
DocuSignTK.TemplateReferenceFieldDataDataValue[2];
templateReference.FieldData.DataValues.DataValue[0] = fd2;
templateReference.FieldData.DataValues.DataValue[1] = fd3;
// Make the call
try {
//DocuSignTK.EnvelopeStatus result = api_sender.CreateAndSendEnvelope(envelope);
// Create draft with all the template information
DocuSignTK.ArrayOfTemplateReference TemplateReferenceArray = new DocuSignTK.ArrayOfTemplateReference();
TemplateReferenceArray.TemplateReference = new DocuSignTK.TemplateReference[1];
TemplateReferenceArray.TemplateReference[0] = templateReference;
DocuSignTK.EnvelopeStatus result = api_sender.CreateEnvelopeFromTemplates( TemplateReferenceArray, recipients, envelopeInfo, true);
envelope_id = result.EnvelopeID;
System.debug('Returned successfully, envelope_id = ' + envelope_id );
} catch ( CalloutException e) {
System.debug('Exception - ' + e );
error_code = 'Problem: ' + e;
error_message = error_code;
}
}
Private Boolean no_error() {
return (String.isEmpty(error_code));
}
}
DocuSign Website uses RESTAPI to send Payment details in an envelope. Payment related APIs are not available in SOAP calls, please try using REST API for payment related calls.

How the prevent Azure table injection?

Is there a general way to prevent azure storage injection.
If the query contains a user entered string for example his name. Then it is possible to do some injection like: jan + ' or PartitionKey eq 'kees.
This will and up getting an object jan and an object with the partitionKey kees.
One option is URLEncoding. In this case ' and " are encoded. And the above injection is not possible anymore.
Is this the best option or are there better ones?
Per my experience, I realize that there is two general ways to prevent azure storage table injection.
The one is replace the string ' with the other string such as ; , " or URLEncode string of '. This is your option.
The other is storage table key using an encoding format(such as Base64) instread of plain content.
This is my test Java program as follows:
import org.apache.commons.codec.binary.Base64;
import com.microsoft.azure.storage.CloudStorageAccount;
import com.microsoft.azure.storage.table.CloudTable;
import com.microsoft.azure.storage.table.CloudTableClient;
import com.microsoft.azure.storage.table.TableOperation;
import com.microsoft.azure.storage.table.TableQuery;
import com.microsoft.azure.storage.table.TableQuery.QueryComparisons;
public class TableInjectTest {
private static final String storageConnectString = "DefaultEndpointsProtocol=http;" + "AccountName=<ACCOUNT_NAME>;"
+ "AccountKey=<ACCOUNT_KEY>";
public static void reproduce(String query) {
try {
CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectString);
CloudTableClient tableClient = storageAccount.createCloudTableClient();
// Create table if not exist.
String tableName = "people";
CloudTable cloudTable = new CloudTable(tableName, tableClient);
final String PARTITION_KEY = "PartitionKey";
String partitionFilter = TableQuery.generateFilterCondition(PARTITION_KEY, QueryComparisons.EQUAL, query);
System.out.println(partitionFilter);
TableQuery<CustomerEntity> rangeQuery = TableQuery.from(CustomerEntity.class).where(partitionFilter);
for (CustomerEntity entity : cloudTable.execute(rangeQuery)) {
System.out.println(entity.getPartitionKey() + " " + entity.getRowKey() + "\t" + entity.getEmail() + "\t"
+ entity.getPhoneNumber());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/*
* The one way is replace ' with other symbol string
*/
public static String preventByReplace(String query, String symbol) {
return query.replaceAll("'", symbol);
}
public static void addEntityByBase64PartitionKey() {
try {
CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectString);
CloudTableClient tableClient = storageAccount.createCloudTableClient();
// Create table if not exist.
String tableName = "people";
CloudTable cloudTable = new CloudTable(tableName, tableClient);
String partitionKey = Base64.encodeBase64String("Smith".getBytes());
CustomerEntity customer = new CustomerEntity(partitionKey, "Will");
customer.setEmail("will-smith#contoso.com");
customer.setPhoneNumber("400800600");
TableOperation insertCustomer = TableOperation.insertOrReplace(customer);
cloudTable.execute(insertCustomer);
} catch (Exception e) {
e.printStackTrace();
}
}
// The other way is store PartitionKey using encoding format such as Base64
public static String preventByEncodeBase64(String query) {
return Base64.encodeBase64String(query.getBytes());
}
public static void main(String[] args) {
String queryNormal = "Smith";
reproduce(queryNormal);
/*
* Output as follows:
* PartitionKey eq 'Smith'
* Smith Ben Ben#contoso.com 425-555-0102
* Smith Denise Denise#contoso.com 425-555-0103
* Smith Jeff Jeff#contoso.com 425-555-0105
*/
String queryInjection = "Smith' or PartitionKey lt 'Z";
reproduce(queryInjection);
/*
* Output as follows:
* PartitionKey eq 'Smith' or PartitionKey lt 'Z'
* Webber Peter Peter#contoso.com 425-555-0101 <= This is my information
* Smith Ben Ben#contoso.com 425-555-0102
* Smith Denise Denise#contoso.com 425-555-0103
* Smith Jeff Jeff#contoso.com 425-555-0105
*/
reproduce(preventByReplace(queryNormal, "\"")); // The result same as queryNormal
reproduce(preventByReplace(queryInjection, "\"")); // None result, because the query string is """PartitionKey eq 'Smith" or PartitionKey lt "Z'"""
reproduce(preventByReplace(queryNormal, "&")); // The result same as queryNormal
reproduce(preventByReplace(queryInjection, "&")); // None result, because the query string is """PartitionKey eq 'Smith& or PartitionKey lt &Z'"""
/*
* The second prevent way
*/
addEntityByBase64PartitionKey(); // Will Smith
reproduce(preventByEncodeBase64(queryNormal));
/*
* Output as follows:
* PartitionKey eq 'U21pdGg='
* U21pdGg= Will will-smith#contoso.com 400800600 <= The Base64 string can be decoded to "Smith"
*/
reproduce(preventByEncodeBase64(queryInjection)); //None result
/*
* Output as follows:
* PartitionKey eq 'U21pdGgnIG9yIFBhcnRpdGlvbktleSBsdCAnWg=='
*/
}
}
I think that the best option is choose a suitable way to prevent query injection on the basis of application sence.
Any concerns, please feel free to let me know.

How to retrieve more than 100 messages from the history of a PubNub channel?

The page about the PubNub History API states that
The history() function returns a list of up to 100 messages, the start
time token and the ending time token.
Is there a way to retrieve more than the 100 messages?
I'm currently not a paying customer of PubNub.
PubNub Load History More than 100 Messages
Sometimes you want to slice back in time over a linear stream of data. And often you'll want to do this at different levels of granularity. That is why PubNub Storage and Playback APIs provide maximum level of flexibility. However sometimes it ends up being a bit tricky to load data with the preferred result set.
PubNub Real-Time Network Storage and Playback
There are several considerations you may be seeking when loading transaction history over timelines that can potentially span millions of message in the transaction set. There are some great options available to you and we will cover two of them right now. The examples will be coded in JavaScript. The first example loads a summary of the data by grabbing the snapshots for the beginning of each hour for the past 24 hours. The second example shows you how to load all transactions in full detail and maximum granularity.
All Reference Files can be found on this GIST: Loading History from PubNub Mt.Gox Trades
Example PubNub Mt.Gox History JavaScript Usage
<script src="https://cdn.pubnub.com/pubnub.min.js"></script>
<script src="mtgox-history.js"></script>
<script>(function(){
// LOAD HOURLY SUMMARY
MTGOX.history.hourly({
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(response) { console.log(JSON.stringify(response)) },
error : function() { console.log("NETWORK ERROR") }
});
// LOAD ALL WITH LIMITER OPTION
MTGOX.history.full({
limit : 500, // SET LIMIT AS HIGH AS NEEDED TO LOAD MORE!
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(messages) { console.log(messages) },
error : function(e) { console.log("NETWORK ERROR") }
});
})();</script>
NOTE: Running MTGOX.history.hourly() method will generate a list of snapshots per hour over the last 24 hours.
NOTE: Running MTGOX.history.full() method will generate maximum resolution detail with a lot of data. You can get a full dump or partial dump as needed; and you should increase the limit parameter in order to grab more data points.
This following JavaScript file will provide you the MTGOX interface.
PubNub Mt.Gox History JavaScript Loader
//
// mtgox-history.js
//
(function(){
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// INITIALIZE PUBNUB
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
var pubnub = PUBNUB.init({
subscribe_key : 'sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe'
});
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// MTGOX HISTORY INTERFACE
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
window.MTGOX = {
history : {
hourly : hourly,
full : full
}
};
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// GET ALL DATA FOREVER (WITH LIMIT OF COURSE)
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/*
MTGOX.history.full({
limit : 1000,
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(messages) { console.log(messages) },
error : function(e) { console.log("NETWORK ERROR") }
});
*/
function full(args) {
var chan = args['channel'] ||'d5f06780-30a8-4a48-a2f8-7ed181b4a13f'
, callback = args['data'] || function(){}
, error = args['error'] || function(){}
, limit = +args['limit'] || 5000
, start = 0
, count = 100
, history = []
, params = {
channel : chan,
count : count,
callback : function(messages) {
var msgs = messages[0];
start = messages[1];
params.start = start;
PUBNUB.each( msgs.reverse(), function(m) {history.push(m)} );
if (history.length >= limit) return callback(history);
if (msgs.length < count) return callback(history);
count = 100;
add_messages();
},
error : function(e) {
callback(history);
error(history);
}
};
add_messages();
function add_messages() { pubnub.history(params) }
}
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
// GET 24 HOURS IN HOURLY INCREMENTS
// -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/*
MTGOX.history.hourly({
channel : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
data : function(response) { console.log(response) },
error : function() { console.log('ERROR') }
});
*/
function hourly(setup) {
var limit = 24;
var count = 0;
var chan = setup['channel'] ||'d5f06780-30a8-4a48-a2f8-7ed181b4a13f';
var cb = setup['data'] || function(){};
var eb = setup['error'] || function(){};
var now = new Date();
now.setUTCHours(0);
now.setUTCMinutes(0);
now.setUTCSeconds(0);
now.setUTCMilliseconds(0);
var utc_now = now.getTime();
var vectors = [];
PUBNUB.each( (new Array(limit)).join(',').split(','), function( _, d ) {
var day = utc_now - 3600000 * d;
pubnub.history({
limit : 1,
channel : chan,
start : day * 10000,
error : function() { count++; eb(); },
callback : function(messages) {
// DONE?
if (++count == limit) return cb(vectors);
// ADD TIME SLICES
var res = +(((messages[0][0]||{}).ticker||{}).avg||{}).value;
res && vectors.push([ new Date(day).getUTCHours(), res ]);
// KEEP IT SORTED
vectors.sort(function(a,b){ return a[0] > b[0] && -1 || 1 });
}
})
} );
}
})();
Mt.Gox PubNub Channel Listing for Tickers, Depth and Trades
The following is a list of channels provided by Mt.Gox data feed options you can use in the history channel parameter field.
{
"TICKER.ltcgbp": "0102A446-E4D4-4082-8E83-CC02822F9172",
"TICKER.ltccny": "0290378C-E3D7-4836-8CB1-2BFAE20CC492",
"DEPTH.btchkd": "049F65DC-3AF3-4FFD-85A5-AAC102B2A579",
"DEPTH.btceur": "057BDC6B-9F9C-44E4-BC1A-363E4443CE87",
"TICKER.nmcaud": "08C65460-CBD9-492E-8473-8507DFA66AE6",
"TICKER.btceur": "0BB6DA8B-F6C6-4ECF-8F0D-A544AD948C15",
"DEPTH.btckrw": "0C84BDA7-E613-4B19-AE2A-6D26412C9F70",
"DEPTH.btccny": "0D1ECAD8-E20F-459E-8BED-0BDCF927820F",
"TICKER.btccad": "10720792-084D-45BA-92E3-CF44D9477775",
"DEPTH.btcchf": "113FEC5F-294D-4929-86EB-8CA4C3FD1BED",
"TICKER.ltcnok": "13616AE8-9268-4A43-BDF7-6B8D1AC814A2",
"TICKER.ltcusd": "1366A9F3-92EB-4C6C-9CCC-492A959ECA94",
"TICKER.btcbtc": "13EDFF67-CFA0-4D99-AA76-52BD15D6A058",
"TICKER.ltccad": "18B55737-3F5C-4583-AF63-6EB3951EAD72",
"TICKER.nmccny": "249FDEFD-C6EB-4802-9F54-064BC83908AA",
"DEPTH.btcusd": "24E67E0D-1CAD-4CC0-9E7A-F8523EF460FE",
"TICKER.btcchf": "2644C164-3DB7-4475-8B45-C7042EFE3413",
"DEPTH.btcaud": "296EE352-DD5D-46F3-9BEA-5E39DEDE2005",
"TICKER.btcczk": "2A968B7F-6638-40BA-95E7-7284B3196D52",
"TICKER.btcsgd": "2CB73ED1-07F4-45E0-8918-BCBFDA658912",
"TICKER.nmcjpy": "314E2B7A-A9FA-4249-BC46-B7F662ECBC3A",
"TICKER.btcnmc": "36189B8C-CFFA-40D2-B205-FB71420387AE",
"DEPTH.btcinr": "414FDB18-8F70-471C-A9DF-B3C2740727EA",
"DEPTH.btcsgd": "41E5C243-3D44-4FAD-B690-F39E1DBB86A8",
"TICKER.btcltc": "48B6886F-49C0-4614-B647-BA5369B449A9",
"TICKER.ltceur": "491BC9BB-7CD8-4719-A9E8-16DAD802FFAC",
"TICKER.btcinr": "55E5FEB8-FEA5-416B-88FA-40211541DECA",
"TICKER.ltcjpy": "5AD8E40F-6DF3-489F-9CF1-AF28426A50CF",
"DEPTH.btccad": "5B234CC3-A7C1-47CE-854F-27AEE4CDBDA5",
"TICKER.btcnzd": "5DDD27CA-2466-4D1A-8961-615DEDB68BF1",
"DEPTH.btcgbp": "60C3AF1B-5D40-4D0E-B9FC-CCAB433D2E9C",
"DEPTH.btcnok": "66DA7FB4-6B0C-4A10-9CB7-E2944E046EB5",
"DEPTH.btcthb": "67879668-532F-41F9-8EB0-55E7593A5AB8",
"TICKER.btcsek": "6CAF1244-655B-460F-BEAF-5C56D1F4BEA7",
"TICKER.btcnok": "7532E866-3A03-4514-A4B1-6F86E3A8DC11",
"TICKER.btcgbp": "7B842B7D-D1F9-46FA-A49C-C12F1AD5A533",
"TRADE.LAG": "85174711-BE64-4DE1-B783-0628995D7914",
"DEPTH.btcsek": "8F1FEFAA-7C55-4420-ADA0-4DE15C1C38F3",
"DEPTH.btcdkk": "9219ABB0-B50C-4007-B4D2-51D1711AB19C",
"DEPTH.btcjpy": "94483E07-D797-4DD4-BC72-DC98F1FD39E3",
"TICKER.nmcusd": "9AAEFD15-D101-49F3-A2FD-6B63B85B6BED",
"TICKER.ltcaud": "A046600A-A06C-4EBF-9FFB-BDC8157227E8",
"TICKER.btcjpy": "A39AE532-6A3C-4835-AF8C-DDA54CB4874E",
"DEPTH.btcczk": "A7A970CF-4F6C-4D85-A74E-AC0979049B87",
"TICKER.ltcdkk": "B10A706E-E8C7-4EA8-9148-669F86930B36",
"TICKER.btcpln": "B4A02CB3-2E2D-4A88-AEEA-3C66CB604D01",
"TEST": "BAD99F24-FA8B-4938-BFDF-0C1831FC6665",
"TICKER.btcrub": "BD04F720-3C70-4DCE-AE71-2422AB862C65",
"TICKER.nmcgbp": "BF5126BA-5187-456F-8AE6-963678D0607F",
"TICKER.btckrw": "BF85048D-4DB9-4DBE-9CA3-5B83A1A4186E",
"TICKER.btccny": "C251EC35-56F9-40AB-A4F6-13325C349DE4",
"DEPTH.btcnzd": "CEDF8730-BCE6-4278-B6FE-9BEE42930E95",
"TICKER.btchkd": "D3AE78DD-01DD-4074-88A7-B8AA03CD28DD",
"TICKER.btcthb": "D58E3B69-9560-4B9E-8C58-B5C0F3FDA5E1",
"TICKER.btcusd": "D5F06780-30A8-4A48-A2F8-7ED181B4A13F",
"DEPTH.btcrub": "D6412CA0-B686-464C-891A-D1BA3943F3C6",
"TICKER.nmceur": "D8512D04-F262-4A14-82F2-8E5C96C15E68",
"TRADE.btc": "DBF1DEE9-4F2E-4A08-8CB7-748919A71B21",
"TICKER.nmccad": "DC28033E-7506-484C-905D-1C811A613323",
"DEPTH.btcpln": "E4FF055A-F8BF-407E-AF76-676CAD319A21",
"TICKER.btcdkk": "E5CE0604-574A-4059-9493-80AF46C776B3",
"TICKER.btcaud": "EB6AAA11-99D0-4F64-9E8C-1140872A423D"
}
See https://help.pubnub.com/entries/24113341-How-do-I-Page-Through-Stored-Messages-
Contact PubNub support (help#...) if further assistance is needed
The below Java class can be used to easily retrieve and process long time ranges of history messages in an advanced for loop of this form:
PubnubHistoryExcerpt history = new PubnubHistoryExcerpt(pubnub, channel, start, end);
for (Object message : history) {
// do something with the message object
}
The messages are retrieved on the fly, so no memory problem occurs.
Below is the complete code. You can find a fully runnable usage example in the main() method inside of the class.
I haven't yet tested this class extensively. Enhancements are welcome.
/*
* PubnubHistoryExcerpt.java
*
* This file is distributed under the FreeBSD License:
*
* Copyright (c) 2014, Daniel S. (http://stackoverflow.com/users/1838726/daniel-s)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are those
* of the authors and should not be interpreted as representing official policies,
* either expressed or implied, of the FreeBSD Project.
*/
import java.math.BigDecimal;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.NoSuchElementException;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import com.pubnub.api.Callback;
import com.pubnub.api.Pubnub;
import com.pubnub.api.PubnubError;
/**
* You can use this class to iterate over historical PubNub messages. The messages are retrieved transparently while you
* iterate over the history excerpt. This class and the returned iterators are thread-safe.
*
* See {#link #main(String[])} for a usage example.
*/
public class PubnubHistoryExcerpt implements Iterable<Object> {
/**
* This main method contains a usage example for this class. It downloads last 3 hour's messages of the MtGox BTC/USD ticker
* channel from PubNub and outputs the timestamp and USD value which are found in the messages.
*/
public static void main(String[] args) throws JSONException {
String PUBNUB_SUBSCRIBE_KEY_MTGOX = "sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe";
String PUBNUB_CHANNEL_MTGOX_TICKER_BTCUSD = "d5f06780-30a8-4a48-a2f8-7ed181b4a13f";
Pubnub pubnub = new Pubnub(null, PUBNUB_SUBSCRIBE_KEY_MTGOX);
long ONE_HOUR_IN_MILLIS = 60 * 60 * 1000;
long end = System.currentTimeMillis();
long start = end - 3 * ONE_HOUR_IN_MILLIS;
// convert from milliseconds as time unit (10^-3 seconds) to
// pubnub's better-than-microsecond precision time units (10^-7 seconds)
start *= 10000;
end *= 10000;
PubnubHistoryExcerpt history = new PubnubHistoryExcerpt(pubnub, PUBNUB_CHANNEL_MTGOX_TICKER_BTCUSD, start, end);
DefaultDateFormat dateFormat = DefaultDateFormat.create();
for (Object message : history) {
JSONObject messageJson = (JSONObject) message;
JSONObject ticker = messageJson.getJSONObject("ticker");
long instant = ticker.getLong("now");
BigDecimal value = new BigDecimal(ticker.getJSONObject("last_local").getString("value"));
instant /= 1000; // convert from microseconds to milliseconds
System.out.println(dateFormat.format(instant) + ": " + value);
}
System.exit(0);
}
/**
* This is the maximum number of messages to fetch in one batch. If you fetch many messages, higher numbers improve
* performance. Setting this to a value higher than 100 doesn't have an effect, because Pubnub currently doesn't
* support fetching more than this many messages at once.
*/
private static final int BATCH_SIZE = 100;
private final Pubnub pubnub;
private final String channel;
private final long start;
private final long end;
/**
* Constructs a new excerpt over which you can iterate. Insances represent an excerpt. No retrieval operations are
* started unless you call iterator().next() for the first time.
*
* #param pubnub
* The Pubnub connection to use for retrieving messages.
* #param channel
* The channel for which to retrieve historical messages.
* #param start
* The beginning of the time interval for which to retrieve messages, in pubnub's time units (10^-7
* seconds, so milliseconds * 10000) since 1970-01-01 00:00:00).
* #param end
* The end of the time interval for which to retrieve messages, in pubnub's time units (10^-7 seconds, so
* milliseconds * 10000) since 1970-01-01 00:00:00).
*/
private PubnubHistoryExcerpt(Pubnub pubnub, String channel, long start, long end) {
this.pubnub = pubnub;
this.channel = channel;
this.start = start;
this.end = end;
}
public Iterator<Object> iterator() {
return new Iter();
}
private class Iter implements Iterator<Object> {
/**
* This list is used as a fifo buffer for messages retrieves through this iterator. It also acts as the main
* synchronization lock for synchronizing access between threads accessing this class as an iterator and threads
* calling back from the Pubnub API.
*/
private LinkedList<Object> buffer = new LinkedList<Object>();
/**
* This field stores the end of the time range of the previous batch retrieval, in Pubnub time units (10th of a
* microsecond, so milliseconds*10000). For the following batch retrieval, this is used as the start time for
* retrieving the following messages.
*/
private long prevBatchTimeRangeEnd = PubnubHistoryExcerpt.this.start;
/**
* Retrieval of messages is handled asynchronously. That means that exceptions which are thrown during retrieval
* can't automatically be propagated through to the code which invokes <code>next()</code> or
* <code>hasNext()</code> . Therefor, such an exception is stored temporarily in this field and then re-thrown
* from within <code>next()</code> or <code>hasNext()</code>.
*/
private Exception caughtDuringRetrieval = null;
/**
* This object is used to wait on and to notify about updates of the buffer.
*/
private Object notifier = new Object();
/**
* Because of spurious wakeups that can happen during wait(), this field is necessary to tell the waiting thread
* if retrieval is still running.
*/
private boolean retrieving = false;
/**
* The callback object to use for retrieving messages. This is stored in a field here for re-use. This is a
* compromise between performance and low memory footprint, slightly in favor of performance.
*/
private InternalCallback internalCallback = new InternalCallback();
private void retrieveNextBatch() {
synchronized (notifier) {
this.retrieving = true;
// String startStr = DefaultDateFormat.create().format(prevBatchTimeRangeEnd / 10000);
// String endStr = DefaultDateFormat.create().format(end / 10000);
// System.out.println("fetching from " + startStr + " till " + endStr);
if (Iter.this.prevBatchTimeRangeEnd < PubnubHistoryExcerpt.this.end) {
PubnubHistoryExcerpt.this.pubnub.history( //
PubnubHistoryExcerpt.this.channel, //
Iter.this.prevBatchTimeRangeEnd, //
PubnubHistoryExcerpt.this.end, //
BATCH_SIZE, //
false, //
Iter.this.internalCallback //
);
waitUntilNextBatchRetrievalFinished();
}
}
}
private void waitUntilNextBatchRetrievalFinished() {
while (this.retrieving) {
try {
this.notifier.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private class InternalCallback extends Callback {
#Override
public void successCallback(String channel, Object message) {
synchronized (Iter.this.notifier) {
try {
processSuccessCallback(channel, message);
} catch (Exception e) {
Iter.this.caughtDuringRetrieval = e;
} finally {
Iter.this.retrieving = false;
Iter.this.notifier.notifyAll();
}
}
}
#Override
public void errorCallback(String channel, PubnubError error) {
Iter.this.caughtDuringRetrieval = new Exception("" + //
error.getClass().getName() + ": " + //
error.getErrorString() + //
" (code=" + error.errorCode + "; extendedCode=" + error.errorCodeExtended + ")");
Iter.this.caughtDuringRetrieval.fillInStackTrace();
}
}
private void processSuccessCallback(String channel, Object message) throws JSONException {
if (message == null)
throw new NullPointerException("retrieved message is null");
if (!(message instanceof JSONArray))
throw new RuntimeException("retrieved message is not a " + JSONArray.class.getName());
JSONArray historyMessage = (JSONArray) message;
// System.out.println(historyMessage.toString(2));
JSONArray messageList = extractMessageList(historyMessage);
long batchTimeRangeEnd = extractBatchTimeRangeEnd(historyMessage);
if (batchTimeRangeEnd > 0)
Iter.this.prevBatchTimeRangeEnd = batchTimeRangeEnd;
else
Iter.this.prevBatchTimeRangeEnd = end;
processMessageList(messageList);
}
private void processMessageList(JSONArray messageList) {
int i = 0;
for (; i < messageList.length(); i++) {
JSONObject message;
try {
message = messageList.getJSONObject(i);
} catch (JSONException e) {
String str;
try {
str = messageList.toString(2);
} catch (JSONException secondaryE) {
str = "(couldn't convert messageList to String because of " + secondaryE.toString() + ")";
}
throw new RuntimeException("couldn't extract message at index " + i + " from messageList (messageList:\n" + str
+ "\n(end of messageList)\n)", e);
}
Iter.this.buffer.add(message);
}
}
private long extractBatchTimeRangeEnd(JSONArray historyMessage) {
long batchTimeRangeEnd;
try {
batchTimeRangeEnd = historyMessage.getLong(2);
} catch (JSONException e) {
String str = safeConvertHistoryMessageToString(historyMessage);
throw new RuntimeException("could not extract element 2 (batchTimeRangeEnd) of retrieved historyMessage (historyMessage:\n" + str
+ "\n(end of historyMessage)\n)", e);
}
return batchTimeRangeEnd;
}
private String safeConvertHistoryMessageToString(JSONArray historyMessage) {
String str;
try {
str = historyMessage.toString(2);
} catch (JSONException secondaryE) {
str = "(couldn't convert historyMessage to String because of " + secondaryE.toString() + ")";
}
return str;
}
private JSONArray extractMessageList(JSONArray historyMessage) {
JSONArray messageArJson;
try {
messageArJson = historyMessage.getJSONArray(0);
} catch (JSONException e) {
String str = safeConvertHistoryMessageToString(historyMessage);
throw new RuntimeException("could not extract element 0 (messageList) of retrieved historyMessage (historyMessage:\n" + str
+ "\n(end of historyMessage)\n)", e);
}
return messageArJson;
}
public boolean hasNext() {
synchronized (Iter.this.buffer) {
ensureNotInExceptionState();
if (Iter.this.buffer.isEmpty())
retrieveNextBatch();
return !Iter.this.buffer.isEmpty();
}
}
public Object next() {
synchronized (Iter.this.buffer) {
if (!hasNext()) {
throw new NoSuchElementException("there are no more elements in this iterator");
}
Object result = Iter.this.buffer.removeFirst();
return result;
}
}
private void ensureNotInExceptionState() {
if (caughtDuringRetrieval != null) {
throw new RuntimeException("an exception was caught already by a previous attempt to access this iterator", caughtDuringRetrieval);
}
}
public void remove() {
throw new UnsupportedOperationException(getClass().getName() + " doesn't support remove()");
}
}
}

In J2ME, How to re-index records in recordstore after deleting any record

I am developing a Location-based J2ME app & in that I'm using RMS to store data.
In RecordStore when I delete any record, the underlying records doesn't get re-indexed. For example, if I have 5 records & I delete record no.2 then record ids will be {1, 3, 4, 5}. But I want record ids after deletion to be {1, 2, 3, 4}. How should I do this??? Because recordId is playing an important role in my app to retrieve & update the record.
You need to change your application logic. ID is just for identification, and not for sorting. Because it is for identification, it must remains the same.
Very often the easiest thing to do is to read and write the whole recordstore at once.
So, since you've said that your record store is basically small (not that much data), I would recommend simply adding your own custom id field to each record. As Meier said, the RMS record id is not really meant to be recalculated, and changed, once a record has been created. So, I would use your own.
If each of your records contain:
boolean isMale
int age
String firstName
then, I would simply add another field at the start of each record:
int id
It makes your records a little bigger, but not much (4 bytes/record). If you'll have less than 64k records, then you could also use a short for the id, and save a couple bytes.
Here's an example (adapted from this IBM tutorial), of reading, writing, and deleting with this kind of record:
private RecordStore _rs;
// these next two methods are just small optimizations, to allow reading and
// updating the ID field in a record without the overhead of creating a new
// stream to call readInt() on. this assumes the id is a 4 byte int, written
// as the first field in each record.
/** Update one record with a new id field */
private static final void putIdIntoRecord(int id, byte[] record) {
// we assume the first 4 bytes are the id (int)
record[0] = (byte)(id >> 24);
record[1] = (byte)(id >> 16);
record[2] = (byte)(id >> 8);
record[3] = (byte)id;
}
/** Get the id field from one record */
private static final int getIdFromRecord(byte[] record) {
// we assume the first 4 bytes are the id (int)
return ((0xFF & record[0]) << 24) |
((0xFF & record[1]) << 16) |
((0xFF & record[2]) << 8) |
(0xFF & record[3]);
}
/** delete a record with the given (custom) id, re-indexing records afterwards */
private void delete(int idToDelete) {
try {
RecordEnumeration enumerator = _rs.enumerateRecords(new IdEqualToFilter(idToDelete),
null, false);
_rs.deleteRecord(enumerator.nextRecordId());
// now, re-index records after 'idToDelete'
enumerator = _rs.enumerateRecords(new IdGreaterThanFilter(idToDelete), null, true);
while (enumerator.hasNextElement()) {
int recordIdToUpdate = enumerator.nextRecordId();
byte[] record = _rs.getRecord(recordIdToUpdate);
// decrement the id by 1
int newId = getIdFromRecord(record) - 1;
// copy the new id back into the record
putIdIntoRecord(newId, record);
// update the record, which now has a lower id, in the store
_rs.setRecord(recordIdToUpdate, record, 0, record.length);
}
} catch (RecordStoreNotOpenException e) {
e.printStackTrace();
} catch (InvalidRecordIDException e) {
e.printStackTrace();
} catch (RecordStoreException e) {
e.printStackTrace();
}
}
/** generate some record store data ... example of writing to store */
public void writeTestData()
{
// just put 20 random records into the record store
boolean[] booleans = new boolean[20];
int[] integers = new int[20];
String[] strings = new String[20];
for (int i = 0; i < 20; i++) {
booleans[i] = (i % 2 == 1);
integers[i] = i * 2;
strings[i] = "string-" + i;
}
writeRecords(booleans, integers, strings);
}
/** take the supplied arrays of data, and save a record for each array index */
public void writeRecords(boolean[] bData, int[] iData, String[] sData)
{
try
{
// Write data into an internal byte array
ByteArrayOutputStream strmBytes = new ByteArrayOutputStream();
// Write Java data types into the above byte array
DataOutputStream strmDataType = new DataOutputStream(strmBytes);
byte[] record;
for (int i = 0; i < sData.length; i++)
{
// Write Java data types
strmDataType.writeInt(i); // this will be the ID field!
strmDataType.writeBoolean(bData[i]);
strmDataType.writeInt(iData[i]);
strmDataType.writeUTF(sData[i]);
// Clear any buffered data
strmDataType.flush();
// Get stream data into byte array and write record
record = strmBytes.toByteArray();
_rs.addRecord(record, 0, record.length);
// Toss any data in the internal array so writes
// starts at beginning (of the internal array)
strmBytes.reset();
}
strmBytes.close();
strmDataType.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
/** read in all the records, and print them out */
public void readRecords()
{
try
{
RecordEnumeration re = _rs.enumerateRecords(null, null, false);
while (re.hasNextElement())
{
// Get next record
byte[] recData = re.nextRecord();
// Read from the specified byte array
ByteArrayInputStream strmBytes = new ByteArrayInputStream(recData);
// Read Java data types from the above byte array
DataInputStream strmDataType = new DataInputStream(strmBytes);
// Read back the data types
System.out.println("Record ID=" + strmDataType.readInt());
System.out.println("Boolean: " + strmDataType.readBoolean());
System.out.println("Integer: " + strmDataType.readInt());
System.out.println("String: " + strmDataType.readUTF());
System.out.println("--------------------");
strmBytes.close();
strmDataType.close();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
Here, I make use of a couple small RecordFilter classes, to use when searching the record store:
/** helps filter out records greater than a certain id */
private class IdGreaterThanFilter implements RecordFilter {
private int _minimumId;
public IdGreaterThanFilter(int value) {
_minimumId = value;
}
public boolean matches(byte[] candidate) {
// return true if candidate record's id is greater than minimum value
return (getIdFromRecord(candidate) > _minimumId);
}
}
/** helps filter out records by id field (not "recordId"!) */
private class IdEqualToFilter implements RecordFilter {
private int _id;
public IdEqualToFilter(int value) {
_id = value;
}
public boolean matches(byte[] candidate) {
// return true if candidate record's id matches
return (getIdFromRecord(candidate) == _id);
}
}

How to get last login details/time for all users?

I am trying to remove the user accounts which are inactive from last 30 days.
I tried fetching User Information List. Checked all of it's properties and fields but coudn't find anything related to last login time.
You can do something like this
public DateTime Get(string attr, string UserName)
{
DomainConfiguration domainConfig = new DomainConfiguration();
using (new SPMonitoredScope("AD Properties"))
{
using (DirectoryEntry domain = new DirectoryEntry("LDAP://" + domainConfig.DomainName, domainConfig.UserName, domainConfig.Password))
{
//DirectorySearcher searcher = new DirectorySearcher(domain, "(|(objectClass=organizationalUnit)(objectClass=container)(objectClass=builtinDomain)(objectClass=domainDNS))");
DirectorySearcher searcher = new DirectorySearcher(domain);
searcher.PageSize = 1000;
searcher.Filter = "(SAMAccountName='" + UserName + "')";
//searcher.Filter = "(|(objectCategory=group)(objectCategory=person))";
searcher.Filter = "(&(objectClass=user) (cn=" + UserName + "))";
var user = searcher.FindOne();
DateTime LastLogon = DateTime.FromFileTime((Int64)user.Properties["lastLogon"].Value);
return LastLogon;
}
}
}
Hope this Helps you.
I do not know why it does gives me the some older dates than i expected.
but at least it will compile and run.
using System.DirectoryServices.AccountManagement;
private static DateTime? GetUserIdFromDisplayName(string displayName)
{
// set up domain context
using (PrincipalContext ctx = new PrincipalContext(ContextType.Domain))
{
// find user by display name
UserPrincipal user = UserPrincipal.FindByIdentity(ctx, displayName);
if (user != null)
{
return user.LastLogon;
}
else
{
return null;
}
}
}

Resources