How do I get all the keys from a namespace? - get

How can I get all the keys from aerospike namespace like I can get keys from a map using map.keyset. I went through this link Aerospike: how do I get record key?. I did what was being told in the answers. I wrote the code with writePolicy.sendKey = true but i am still getting NULL when I display the key in scanCallback funtion.
Here is my code to write a record (any help would be appreciated)
def writeToAerospike(): Boolean = {
val host:Host = new Host("xx.xxx.xxx.xx", 3000)
val client = new AerospikeClient(new ClientPolicy,host)
val policy = new WritePolicy();
policy.sendKey = true;
policy.timeout = 50000;
if(client.isConnected()){
println("connection to aerospike client sucessful")
client.put(policy,new Key("test", "testSet", "1"),new Bin("name", "john"))
//verify if the record is entered correctly
println(client.get(policy,new Key("test", "testing", "1")).getValue("name"))
client.close()
return true
}
return false
}
I am using the code from this example to scan the records: http://www.aerospike.com/docs/client/java/usage/scan/scan.html
class ScanParallelTest extends ScanCallback {
private var recordCount: Int = _
def runTest() {
val client = new AerospikeClient("xxx.xx.xxx.xx", 3000)
try {
val policy = new ScanPolicy()
policy.concurrentNodes = true
policy.priority = Priority.LOW
policy.includeBinData = true
client.scanAll(policy, "test", "testSet", this)
println("Records " + recordCount)
} finally {
client.close()
}
}
def scanCallback(key: Key, record: Record) {
recordCount += 1
println("Records " + recordCount )
println ("key=" + " "+key.userKey.toString() +" and "+" record: "+" "+ record.bins)
}
}
Here is the error I am getting:
com.aerospike.client.AerospikeException: java.lang.NullPointerException
at com.aerospike.client.command.ScanExecutor.scanParallel(ScanExecutor.java:66)
at com.aerospike.client.AerospikeClient.scanAll(AerospikeClient.java:551)
at aerospike.asd.ScanParallelTest.runTest(ScanParallelTest.scala:23)
at aerospike.asd.test$.main(test.scala:10)
at aerospike.asd.test.main(test.scala)
Caused by: java.lang.NullPointerException
at aerospike.asd.ScanParallelTest.scanCallback(ScanParallelTest.scala:35)
at com.aerospike.client.command.ScanCommand.parseRecordResults(ScanCommand.java:122)
at com.aerospike.client.command.MultiCommand.parseResult(MultiCommand.java:58)
at com.aerospike.client.command.SyncCommand.execute(SyncCommand.java:56)
at com.aerospike.client.command.ScanExecutor$ScanThread.run(ScanExecutor.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I tried using client.readPolicyDefault.sendKey = true; and client.scanPolicyDefault.sendKey = true; with client.writePolicyDefault.sendKey = true; and the code started to return the original keys instead of NULL.
Here is a link to a working code by helipilot50: https://github.com/helipilot50/store-primary-key
It really helped in my case. Thank you!

Adding to Fahad's answer:
Keep in mind that storing the key (rather than just the Digest) takes up space. If you key is a 100 character string and your record is two 8 byte integers, the key storage will use more space than the the two values.
Use this knowledge for good and not for evil :)

Related

is it possible to connect to java jOOQ DB?

I discovered a new interesting service and I'm trying to understand how it works. Please explain how to connect to my jOOQ database from another program?
MockDataProvider provider = new MyProvider();
MockConnection connection = new MockConnection(provider);
DSLContext create = DSL.using(connection, SQLDialect.H2);
Field<Integer> id = field(name("BOOK", "ID"), SQLDataType.INTEGER);
Field<String> book = field(name("BOOK", "NAME"), SQLDataType.VARCHAR);
So, I create but can I connect to it?
Here I have added your code, Lukas.
try (Statement s = connection.createStatement();
ResultSet rs = s.executeQuery("SELECT ...")
) {
while (rs.next())
System.out.println(rs.getString(1));
}
This example was found here
https://www.jooq.org/doc/3.7/manual/tools/jdbc-mocking/
public class MyProvider implements MockDataProvider {
#Override
public MockResult[] execute(MockExecuteContext ctx) throws SQLException {
// You might need a DSLContext to create org.jooq.Result and org.jooq.Record objects
//DSLContext create = DSL.using(SQLDialect.ORACLE);
DSLContext create = DSL.using(SQLDialect.H2);
MockResult[] mock = new MockResult[1];
// The execute context contains SQL string(s), bind values, and other meta-data
String sql = ctx.sql();
// Dynamic field creation
Field<Integer> id = field(name("AUTHOR", "ID"), SQLDataType.INTEGER);
Field<String> lastName = field(name("AUTHOR", "LAST_NAME"), SQLDataType.VARCHAR);
// Exceptions are propagated through the JDBC and jOOQ APIs
if (sql.toUpperCase().startsWith("DROP")) {
throw new SQLException("Statement not supported: " + sql);
}
// You decide, whether any given statement returns results, and how many
else if (sql.toUpperCase().startsWith("SELECT")) {
// Always return one record
Result<Record2<Integer, String>> result = create.newResult(id, lastName);
result.add(create
.newRecord(id, lastName)
.values(1, "Orwell"));
mock[0] = new MockResult(1, result);
}
// You can detect batch statements easily
else if (ctx.batch()) {
// [...]
}
return mock;
}
}
I'm not sure what lines 3-5 of your example are supposed to do, but if you implement your MockDataProvider and put that into a MockConnection, you just use that like any other JDBC connection, e.g.
try (Statement s = connection.createStatement();
ResultSet rs = s.executeQuery("SELECT ...")
) {
while (rs.next())
System.out.println(rs.getString(1));
}

Cassandra Java Exception - NoHostAvailableException

I have setup single Cassandra node on VM. i have to create a table with 70000 columns. for this i have written java code that read json file and create table.
here is my java code snippet.
When i run my java code it throws exception after creation some columns.
Exception stack is
public void createTable(String keyspaceName, String tableName) throws FileNotFoundException{
JSONParser jsonParser = new JSONParser();
FileReader fileReader;
String filePath = "";
String columnHeader = "";
//String completeColumnHeader = "";
try{
System.out.println("Inside Create Table");
session.executeAsync("DROP TABLE IF EXISTS "+keyspaceName+"."+tableName+";");
String createQuery = "CREATE TABLE "+keyspaceName+"."+tableName +"(\"P:LanguageID\" text, "
+ "\"P:PdmarticleID\" text, PRIMARY KEY(\"P:PdmarticleID\",\"P:LanguageID\"));";
session.execute(createQuery);
System.out.println("Table created");
filePath = "CassandraTableColumnHeader/FixColumnHeader.json";
fileReader = new FileReader(filePath);
JSONObject jsonObject = (JSONObject) jsonParser.parse(fileReader);
JSONArray jsonArray = (JSONArray) jsonObject.get("columnHeaderName");
int columnHeaderSize = jsonArray.size();
int columnHeaderBatchSize = 1000;
int fromIndex = 0;
int toIndex = columnHeaderBatchSize;
while(columnHeaderSize > 0){
columnHeaderSize -=columnHeaderBatchSize;
for(int i = fromIndex; i < toIndex; i++) {
columnHeader = (String) jsonArray.get(i);
if(columnHeader.equals("P:PdmarticleID")||columnHeader.equals("P:LanguageID")){
continue;
}
session.execute("ALTER TABLE "+keyspaceName+"."+tableName +" ADD "+"\""+columnHeader+"\""+" text;");
}
fromIndex = toIndex;
if(columnHeaderSize < columnHeaderBatchSize){
toIndex += columnHeaderSize;
}else{
toIndex = toIndex + columnHeaderBatchSize;
}
}
}catch(FileNotFoundException fnfe){
throw fnfe;
}catch (ParseException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Exception in thread "main" com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.DriverException: Host replied with server error: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException: C:\apache-cassandra-new\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmplink-ka-4839-Data.db (The process cannot access the file because it is being used by another process)))
at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:265)
at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:179)
at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
at com.exportstagging.SparkTest.DataLoaderInCassandra.createTable(DataLoaderInCassandra.java:89)
at com.exportstagging.SparkTest.DataLoaderInCassandra.main(DataLoaderInCassandra.java:216)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.DriverException: Host replied with server error: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException: C:\apache-cassandra-new\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmplink-ka-4839-Data.db (The process cannot access the file because it is being used by another process)))
at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:216)
at com.datastax.driver.core.RequestHandler.access$900(RequestHandler.java:45)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution.sendRequest(RequestHandler.java:276)
at com.datastax.driver.core.RequestHandler$SpeculativeExecution$1.run(RequestHandler.java:374)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
I have stuck here. Please help me. Thanks in advance.
If I were you I might reevaluate creating a table with 70k column headers. Your partition key P:PdmarticleID and full primary key (P:PdmarticleID, P:LanguageID) are the only two pieces of information you will be able to use to get results anyway. So having these other pieces of information explicitly stored in columns is not buying you anything.
A collection (eg. map) can hold onto 64k items, with certain other limitations (see http://wiki.apache.org/cassandra/CassandraLimitations). Is there a way you can split the columns such that you can create multiple tables, with some pieces of information stored in one table and some in another?

Why is the scanner not finding a line after the first iteration?

I'm writing a program that basically acts as an email client as a part of a homework assignment for a class in java. I don't normally resort to the internet to answer my questions, but this is something that goes beyond what the professor is trying to get us to learn/practice, and I need to find a way to fix it.
The Problem: If I run the program, and input the second command (ri), it will prompt me for the number, and then finish by displaying the message, but immediately after, when it goes back into the second iteration of the .run() method the console returns:
Exception in thread "main" java.util.NoSuchElementException: No Line Found
at java.util.scanner.nextLine(Unknown Source)
Here's the code (I'm only including the important stuff...or at least what I think is important).
public class CmdLoop {
private MailClient _client;
Scanner kbd;
private Hashtable<String, ICommand> _commands = new Hashtable<String, ICommand>();
public CmdLoop(MailClient client) {
_client = client;
_commands.put("h", new client.cmd.Help());
_commands.put("ri", new client.cmd.ReadInbox());
kbd = new Scanner(System.in);
}
public void run2() {
System.out.print("\nMail: ");
String command = kbd.nextLine();
ICommand call = _commands.get(command);
if (command.equals("q"))
return;
else if (call == null)
System.out.println(command + " not understood, type h for help");
else if (call.equals(""))
System.out.println(command + " not understood, type h for help");
else call.run(_client);
this.run2();
}
and the ri class:
public class ReadInbox implements ICommand {
#Override
public void run(MailClient client) {
Scanner sc = new Scanner(System.in);
MailBox in = client.getInbox();
if(in.count() < 1)
System.out.println("Inbox empty");
else {
System.out.print("Enter the number of the message you would like to read: ");
int n = Integer.parseInt(sc.nextLine());
if(n > in.count())
System.out.println("Message number " + n + " can't be found");
else
in.getMessage(n - 1).show();
}
sc.close();
}
}
Basically it gets to the end of the ReadInbox.run() call, then it calls this.run2(), returns to the top, outputs "Mail: " and then returns the error. This is what the console looks like in my test:
Mail: ri
Enter the number of the message you would like to read: 1
Date: 2015/04/29 20:24:17
From: Charles Barkley (Charlie) <Charles#gmail.com>
Subj: testerino
this is another test
Mail: Exception in thread "main" java.util.NoSuchElementException: No line found
at java.util.Scanner.nextLine(Unknown Source)
at client.CmdLoop.run2(CmdLoop.java:28)
at client.CmdLoop.run2(CmdLoop.java:37)
at Main.main(Main.java:29)
If I'm correct, getting it so they both operate off the same scanner would solve my problem, and I feel like I should know how to do that, but I'm drawing a blank. Is there another way as well?
According to the documentation a Scanner will throw that exception "if no line was found". You can prevent that by first calling kbd.hasNextLine() which will tell you if there is something to get. That will wait for a line to be entered as long as the scanner is not closed.
Try this in your run2 method:
String command = null;
if (kbd.hasNextLine())
command = kbd.nextLine();
ICommand call = _commands.get(command);
I found the answer. I got rid of sc.close(); in the ReadInbox class. I have 0 clue why this works, but it does. If anyone has an explanation to offer, much appreciated, otherwise, whatever, at least it works.

XAdES-BES countersignature cannot resolve reference error

I implemented verification for XAdES-BES and after testing everything works now besides counter-signatures. The same error occurs not only for files signed with xades4j but also using other software so it is not related to any mistakes in my countersignature implementation. I wonder if I should implement additional ResourceResolver? I added a countersigned file as the attachment with 'REMOVED' for some private entries here.
Below is the code for verification. certDataList is a list with all certificates from the document in String and getCert will return List. DummyCertificateValidationProvider returns ValidationData with a list of previously constructed x509certs.
public boolean verify(final File file) {
if (!Dictionaries.valid()) {
return true;
}
certList = null;
try {
final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setNamespaceAware(true);
final DocumentBuilder db = dbf.newDocumentBuilder();
final Document doc = db.parse(file);
doc.getDocumentElement().normalize();
final NodeList nList = doc.getElementsByTagName("ds:Signature");
Element elem = null;
for (int temp = 0; temp < nList.getLength(); temp++) {
final Node nNode = nList.item(temp);
if (nNode.getNodeType() == Node.ELEMENT_NODE) {
elem = (Element) nNode;
}
}
final NodeList nList2 = doc.getElementsByTagName("ds:X509Certificate");
final List<String> certDataList = new ArrayList<String>();
for (int temp = 0; temp < nList2.getLength(); temp++) {
final Node nNode = nList2.item(temp);
certDataList.add(nNode.getTextContent());
}
certList = getCert(certDataList);
final CertificateValidationProvider certValidator = new DummyCertificateValidationProvider(certList);
final XadesVerificationProfile p = new XadesVerificationProfile(certValidator);
final XadesVerifier v = p.newVerifier();
final SignatureSpecificVerificationOptions opts = new SignatureSpecificVerificationOptions();
// for relative document paths
final String baseUri = "file:///" + file.getParentFile().getAbsolutePath().replace("\\", "/") + "/";
LOGGER.debug("baseUri:" + baseUri);
opts.useBaseUri(baseUri);
v.verify(elem, opts);
return true;
} catch (final IllegalArgumentException | XAdES4jException | CertificateException | IOException | ParserConfigurationException | SAXException e) {
LOGGER.error("XML not validated!", e);
}
return false;
}
Here is the stacktrace:
21:31:48,203 DEBUG ResourceResolver:158 - I was asked to create a ResourceResolver and got 0
21:31:48,203 DEBUG ResourceResolver:101 - check resolvability by class org.apache.xml.security.utils.resolver.ResourceResolver
21:31:48,204 DEBUG ResolverFragment:137 - State I can resolve reference: "#xmldsig-5de7b1d0-be70-4dde-b746-3f4d4d6de39f-sigvalue"
21:31:48,204 ERROR SignComponent:658 - XML not validated!
xades4j.XAdES4jXMLSigException: Error verifying the signature
at xades4j.verification.XadesVerifierImpl.doCoreVerification(XadesVerifierImpl.java:284)
at xades4j.verification.XadesVerifierImpl.verify(XadesVerifierImpl.java:188)
at com.signapplet.sign.SignComponent.verify(SignComponent.java:655)
...
Caused by: org.apache.xml.security.signature.MissingResourceFailureException: The Reference for URI #xmldsig-5de7b1d0-be70-4dde-b746-3f4d4d6de39f-sigvalue has no XMLSignatureInput
Original Exception was org.apache.xml.security.signature.ReferenceNotInitializedException: Cannot resolve element with ID xmldsig-5de7b1d0-be70-4dde-b746-3f4d4d6de39f-sigvalue
Original Exception was org.apache.xml.security.utils.resolver.ResourceResolverException: Cannot resolve element with ID xmldsig-5de7b1d0-be70-4dde-b746-3f4d4d6de39f-sigvalue
at org.apache.xml.security.signature.Manifest.verifyReferences(Manifest.java:414)
at org.apache.xml.security.signature.SignedInfo.verify(SignedInfo.java:259)
at org.apache.xml.security.signature.XMLSignature.checkSignatureValue(XMLSignature.java:724)
at org.apache.xml.security.signature.XMLSignature.checkSignatureValue(XMLSignature.java:656)
at xades4j.verification.XadesVerifierImpl.doCoreVerification(XadesVerifierImpl.java:277)
... 39 more
Caused by: org.apache.xml.security.signature.ReferenceNotInitializedException: Cannot resolve element with ID xmldsig-5de7b1d0-be70-4dde-b746-3f4d4d6de39f-sigvalue
Edit:
The same error occurs when I try to validate file provided with xades4j unit tests document.signed.bes.cs.xml.
Caused by: org.apache.xml.security.signature.MissingResourceFailureException: The Reference for URI #xmldsig-281967d1-74f8-482c-8222-ed58dbd1909b-sigvalue has no XMLSignatureInput
Original Exception was org.apache.xml.security.signature.ReferenceNotInitializedException: Cannot resolve element with ID xmldsig-281967d1-74f8-482c-8222-ed58dbd1909b-sigvalue
Caused by: org.apache.xml.security.signature.ReferenceNotInitializedException: Cannot resolve element with ID xmldsig-281967d1-74f8-482c-8222-ed58dbd1909b-sigvalue
The problem was with ds:Signature. In counter signatures you will have more than one ds:Signature entry. In my verification method I used the for loop:
for (int temp = 0; temp < nList.getLength(); temp++) {
final Node nNode = nList.item(temp);
if (nNode.getNodeType() == Node.ELEMENT_NODE) {
elem = (Element) nNode;
}
}
As you can see, there was no break when element was found so I ended up with the last ds:Signature, not the first one so all previous signatures could not be found.

how to run stored procedure from groovy that returns multiple resultsets

I couldnt find any good example of doing this online.
Can someone please show how to run a stored procedure (that returns multiple resultsets) from groovy?
Basically I am just trying to determine how many resultsets the stored procedure returns..
I have written a helper which allows me to work with stored procedures that return a single ResultSet in a way that is similar to working with queries with groovy.sql.Sql. This could easily be adapted to process multiple ResultSets (I assume each would need it's own closure).
Usage:
Sql sql = Sql.newInstance(dataSource)
SqlHelper helper = new SqlHelper(sql);
helper.eachSprocRow('EXEC sp_my_sproc ?, ?, ?', ['a', 'b', 'c']) { row ->
println "foo=${row.foo}, bar=${row.bar}, baz=${row.baz}"
}
Code:
class SqlHelper {
private Sql sql;
SqlHelper(Sql sql) {
this.sql = sql;
}
public void eachSprocRow(String query, List parameters, Closure closure) {
sql.cacheConnection { Connection con ->
CallableStatement proc = con.prepareCall(query)
try {
parameters.eachWithIndex { param, i ->
proc.setObject(i+1, param)
}
boolean result = proc.execute()
boolean found = false
while (!found) {
if (result) {
ResultSet rs = proc.getResultSet()
ResultSetMetaData md = rs.getMetaData()
int columnCount = md.getColumnCount()
while (rs.next()) {
// use case insensitive map
Map row = new TreeMap(String.CASE_INSENSITIVE_ORDER)
for (int i = 0; i < columnCount; ++ i) {
row[md.getColumnName(i+1)] = rs.getObject(i+1)
}
closure.call(row)
}
found = true;
} else if (proc.getUpdateCount() < 0) {
throw new RuntimeException("Sproc ${query} did not return a result set")
}
result = proc.getMoreResults()
}
} finally {
proc.close()
}
}
}
}
All Java classes are usable from Groovy. If Groovy does not give you a way to do it, then you can do it Java-way using JDBC callable statements.
I just stumbled across what could possibly be a solution to your problem, if an example was what you were after, have a look at the reply to this thread

Resources