I am trying to query Cassandra using Apache Drill. The only connector I could find is here:
http://www.confusedcoders.com/bigdata/apache-drill/sql-on-cassandra-querying-cassandra-via-apache-drill
However this does not build. It comes up with an artifact not found error. I also had another developer who is more versed in these tools take a stab at it, but he also had no luck.
I tried contacting the developer of the plugin I referenced, but the blog does not work and won't let me post comments. Has anyone got this plugin to work (if so how?) or is there another plugin or method I can use to connect apache drill to Cassandra? If anyone could show me how to connect an execute a simple SQL query that would be much appreciated.
I looked at the latest Cassandra storage plugin patch and the latest apache drill source. The drill code has changed and the patch can no longer be applied.
I then manually took the patch apart (it id mostly diff output). Most of the patch was new classes which I could easily add to the latest drill source tree. Most of the other updates were easy to insert into the current source. There were two specific classes that required some minor code modifications/extensions. I rebuilt the distribution from the modified source and installed the drill servers it on a 3 node cluster. The Cassandra schema failed to initialize properly throwing a null pointer exception one of the new classes. This leads me to believe that the (latest) modifed storage plugin is incompatible with the latest version of Cassandra. Since the author of the original storage plugin is unreachable and no one else is stepping up to support the code, this is a dead horse. Beat it if you must.
I was the author of the patch written a year back. Could not get it merged into Drill then, and later got occupied with other stuffs :(
With so many changes to Drill internals, I am not sure what amount of welding would be needed at this point to get it working. Please use the code just as a reference for writing a Drill storage plugin.
Have added this banner on top of the blog post to save fellow developer's hours.
I don't know if anyone is still interested in this topic but I've been experimenting with this plugin and got it to work with Drill 1.18-SNAPSHOT. Here is a link to my branch with this code: 1. My plan is to submit this as a PR for Drill, but it still needs some work. This code will successfully query Cassandra 3.11.5 (latest stable version).
Related
I have a big database of activities which in the LCI have activities from an old ecoinvent version.
Is there a fast and easy way to update all the LCIs in the database so that they use a newer version of ecoinvent?
I suppose there might be problems of activities not existing anymore/changed names, if that can be a case, how do you manage that?
I'm new to Brightway so I have no idea about how to do this ;).
Thanks a lot!
Daina
There isn't yet, but we are working on it. What we normally recommend is to have a reproducible workflow, using whatever data source you prefer. It is very helpful to be able to reproduce your projects if something happens to your computer, and in this case you could re-create your own data and link it to a newer version of whatever database. But you are right, things change from version to version, and this normally requires some metadata on how to handle these changes - hence the library under development.
I need to upgrade my solr search from 4.7 version to 5.3.1 .
I am working on a linux platform.
Can you please provide me the steps that i need to follow .
Thank you!
I do not think that there is a definitive step by step guide for the upgrade that you are looking for.
I have a 4.3.x SOLR running in a production environment and I am contemplating the leap to upgrade to 5.x. However its clear that a lot has changed and that my upgrade is not going to be straight forward.
Also other priorities in my project have kept me from doing the upgrade.
So rest of the discussion is more a thought process than actual upgrade experience.
Last I researched I found the below links useful
https://support.lucidworks.com/hc/en-us/articles/203776523-How-to-upgrade-between-major-Solr-Versions
https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+4+to+Solr+5
From the Major changes link you will notice that a lot has changed ..
Most notably there are changes to the index format, SolrJ removal of deprecated API and that the deployment is now as a standalone server instead of a war file.
So I would suggest that you ask yourself the following questions ...
Is it possible to recreate the index from scratch ? How much time does it take to create your complete index ? If your index can be recreated quickly then , I would suggest that you do that using 5.x engine on a separate machine, while your production environment is served by your existing server. Then plan a complete upgrade from 4.x to 5.x by simply pointing your Production instance to the new SOLR engine. This approach will give you a clean slate to start with and a brand new index (but with existing data).
If you have a very large index (e.g. it takes several days to recreate it from scratch), then you may want to perform an upgrade of the live index. In that case I suggest that you consider the following.
The SOLR upgrade guide mentions 4.10 as a version that is 4.x (so I assume its is easy to upgrade from any 4.x to 4.10) and has some features built in to help with the move to 5.x. So first upgrade to 4.10 ensure that your index continues to work properly. Then use the guides mentioned above to upgrade to 5.x
I've been trying to work through an issue that developed while attempting to upgrade our testing environment from 12.04 to 14.04 ubuntu on aws. Prior to this, the Package repository version of solr was 1.4.1 which matched our 1.4.1 solrj client integrated with our application.
Changing the base AMI to the 14.04 latest and running our default deploy caused solr 3.6.2 server to be installed. It appears it was accepting our configs without issue, however when our client tried to connect we received different errors:
The first was an unknown custom field, which we traced back to our deployment scripts not moving our schema.xml and solrconfig.xml to /etc/solr/conf/ but keeping it in the base directory.
We corrected this issue, and then ran into the following:
'exception: Invalid version or the data in not in 'javabin' format'
This was generated by a wrapper ontop of solrj, but I'll be honest and say I know nothing regarding Solr and that this may be on our end. I've asked our dev team to look at 2 options:
1) enabling: 'server.setParser(new XMLResponseParser());'
Which is the recommendation on the backwards compatibility for an older client.
2) updating our client in the application to 3.6.2
-I know less about the requirements on this.
My fall back is to revert to 1.4.1, but it appears it hasn't been touched since 2011, which makes me hesitant.
Any thoughts / suggestions would be appreciated!
Thanks!
I think the best option is to maintain the same version of Solr and Solrj.
I used for a lot of time Solr 1.4.1 and, while as you said, the most part of it works with newer versions without any problem, actually a lot of things have been changed since 1.4.*
I did your same porting last year, (from 1.4.1 to 3.6.1) and I can confirm you that the 2nd way is the right one: all changes you must do in your client code are just "formal" and very very quick.
Any workaround you could do for being able to communicate with a different version (between Solrj and Solr) is just, as the word says, a "workaround" and it could lead to unexpected (hidden) side-effects later.
I want to know how to apply the issue changes done in the Liferay Issues can be applied in our portal.
For example my issues are cleared in the following links,
https://issues.liferay.com/browse/LPS-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
https://issues.liferay.com/browse/LPS-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
I want to apply the issue changed in the above links in my portal.
Please some one help me to achieve this.
Thanks in Advance.
Regards,
Dinesh M K
You have to see the resolution and fix version stated.
the second issue says :
Fix Version/s: --Sprint 12/11, 6.1.0 CE RC1
and that means that the issue was solved in all Portals after version 6.1.0 CE.
the first issue , is no longer reproducible, and duplicate of https://issues.liferay.com/browse/LPS-14220 which is again solved in 6.1.0 CE RC1
In other words, If your portal is older than this version, you can't do anything. You'll have to upgrade to version 6.1.0 RC1 or later
It seems that the fixed-in version is a bit weirdly set in those issues - e.g. LPS-14220 is a subtask of a story in LPS-14414, which is stated as completed for 6.2.0. Careful: I did not fully read through all of the issue's descriptions, links and mix what I read with the answer (and comment to) #yannicuLar gave
Basically, this seems to be a new feature. The way to backport it to your installation is to identify the relevant commits (e.g. download the repository from https://github.com/liferay/liferay-portal or https://github.com/liferay/liferay-plugins, identify the relevant commits (they all contain the LPS number), "backport" them (e.g. see if they can just be applied to your codebase or if they need manual adaptation because the whole code changed.
Some features are easier to backport than others, I can't tell about the complexity for this one.
In order to separate your changes from Liferay's core changes, you should try to implement this in plugins (or patched plugins) rather than changing the original code and recompile. Most likely it's only the kaleo-web plugin that's effected, but if there are core changes, you'd be better of having them isolated in plugins.
The simplest (and most futureproof) possibility is to wait a bit for 6.2 (RCs are already out) and upgrade your portal to this version. If you want to stay on a version that gets updates, you should do this soon anyway.
I have a remote operations tool made in python and I have distributed it to my friend and he is using it good.
Now, whenever I add a feature to my app i would have to text him the link of my app (manual update type thing) which is bugging for a while.
So is there anyway I could add a auto-update feature a.k.a. check for update to my application for a automated update???
I googled about this but I couldn't find anything related and so I am stuck
Thank you in advance
There is a library called esky in which if you freeze with it, it can check for updates on a certain website, get a list of avaliable versions on a certain website, distribute patch files, and much more. check out here: esky p.s. I dont know if there is a version for 3.x though.