My understanding is that in 2.X everything happens at the manage.py stage with haystack and therefore site.register is no longer required. Some of the register statements in the codebase have been guarded with the check for settings.TESTING and some have not. How can I effect that functionality with haystack 2.X?
I don't have the reputation to comment (this may not be a real answer). I haven't used the old site.register stuff in haystack. I'm also guessing a little at what you are trying to do. With those caveats...
One possibility is to have two indexes, one for testing and one as your default. Do you include or exclude based on settings.TESTING being true? The migration docs mention an EXCLUDED_INDEXES setting that you can use in your backend definitions. So maybe if you have some expensive/unneeded indexes, these can go in a separate search index file/module and then be excluded from your testing backend. The ones in your testing and default index could use other indexes in a common file.
With the above, you would also need to mess with the routing so the right backend gets used during tests. Another possibility is to just define the "real" default in your main settings. Then, in a test_settings file, override the haystack connection with your testing setup as the default (including the EXCLUDED_INDEXES). Then you don't need to sprinkle your tests with using('test') 's.
https://django-haystack.readthedocs.org/en/latest/multiple_index.html might be good reading if you haven't already looked at it 80 times trying to get your migration to work ;)
Related
I am currently working on an Elixir/Phoenix project and I was wondering what is a good way to create a quick in-memory search index.
The index would be created on request and destroyed when the request is over and currently the data comes from a database via Ecto. Also, I would like to query it by different indexes so not just by :id but other indexes Example :user_id so a flat key value store may not be enough.
Are there any tools that would be helpful? I looked a bit into mnesia but when using it with ecto3_mnesia, a local file/folder was created and I would prefer if everything was in memory.
Thanks
I have no idea about ecto3_mnesia, but I am pretty sure raw :mnesia without any redundant wrapper is a good fit here (or, even, :ets if you don’t need a clustered solution.)
:mnesia.table_create/2 accepts many options, two you might be interested in are disc_copies and raw_copies. Simply initialize the former with empty node list and the latter with your complete node list, and you are all set: no disk copies are created, everything is in memory.
I did a google search, but I could not find what I really need.
I need to query an API, which have the same route, but with different parameters.
Example:
router.get('/items/:query, function(){})
In this case, I would search for all items
router.get('/items/:id, function(){})
Here, I would look for a specific item
At the core of your issue is that you are trying to specify two different resources at the same location. If you design your API to adhere to restful principles you'll see why that's not a wise choice. Here are two good starting points:
https://en.wikipedia.org/wiki/Representational_state_transfer
http://www.restapitutorial.com/lessons/whatisrest.html
In restful api's the root resource represents the item collection:
/api/items
...and when you specify an id that indicates you want only one item:
/api/items/abc123
If you really still want to accomplish what you asked in your question you'll need to add a url parameter like /items/query?query=true or /items/abc123?detail=true but this will be confusing to 99% of web developers who ever look at your code.
Also I'm not sure if you really meant this, but when you pass a variable named "query" to the server that seems to indicate that you're going to send a SQL query (or similar data definition language) from the client into the server. This is a dangerous practice for several reasons - it's best to leave all of this type of code on your server.
Edit: if you really absolutely positively have to do it this way then maybe have a query parameter that says ?collection=true. This would at least be understood by other developers that might have to maintain the code in future. Also make sure you add comments to explain why you weren't able to implement rest so you're not leaving behind a bad reputation :)
The issue is that without additional pattern matching there isn't a way Express will be able to distinguish between /items/:query and /items/:id, they are the same route pattern, just with different aliases for the parameter.
Depending on how you intend to structure your query you may want to consider having the route /items and then use query string parameters or have a separate /items/search/:query endpoint.
Does anyone know if there's a recommended set of database indexes for Orchard's core modules? I can't seem to find much info around this, and while I appreciate that the code uses NHibernate to abstract the underlying database, I suspect 99% of users will simply be using SQL server/Express as the default DB and would require suitable indexes to be added. For example on: Orchard_Framework_ContentItemVersionRecord ([Published], [ContentItemRecord_id])
If there isn't already, would it be a good idea for the core modules to have a recommended set of indexes documented somewhere, as they're clearly going to be required for any serious deployments based on an RDBMS?
You're right that some indices would help queries. However with indices it really depends on the usage pattern so there are not many built-in; but you can always add them yourself depending on your specific usage (you can also use SQL Server's tools to recommend indices for your). You can even add indices from migrations to other modules' tables.
In the latest source of Orchard (not yet released) there are also some more default indicies, also for ContentItemVersionRecord. You can see them in FrameworkDataMigration in the 1.x branch.
I need to replicate in CouchDB data from one database to another but in the process I want to alter the documents being replicated over,
mostly stripping out particular fields (but other applications mentioned in comments).
The replication would always be 100% one way (but other applications mentioned in comments could use bi-directional and sync)
I would prefer if this process did not increment their revision ID but that might be asking for too much.
But I don't see any of the design document functions that do what I am trying to do.
As it seems doesn't do this, what plans are there for adding this? And meanwhile, what workarounds are there?
No, there is no out-of-the-box solution, as this would defy the whole purpose and logic of multi-master, MVCC logic.
The only option I can see here is to create your own solution, but I would not call this a replication, but rather ETL (Extract, Transform, Load). And for ETL there are tools available that will let you do the trick, like (mixing open source and commercial here):
Scriptella
CloverETL
Pentaho Data Integration, or to be more specific Kettle
Jespersoft ETL
Talend have some tools as well
There is plenty more of ETL tools on the market.
I believe the best approach here would be to break out the fields you want to filter out into a separate document and then filter out the document during replication.
Of course the best way would be to have built-support for this, but a workaround which occurs to me would be, instead of here using the built-in replication, to code and use a custom replication which will do the additional needed alterations/transformations, still using rather than going beneith, the other built-ins, and with good coding, in many situations (especially if each master can push to its slaves), it feels this could be nearly as efficient.
This requires efficient triggers be put on each source/master to detect any changes, which I believe CouchDB does offer (or at least PouchDB appears to), which would then copy the changes to another location also doing the full alterations.
If the source of the change is unable to push the change to the final destination, this fixed store may to be local to it where the destination can pull from -- which could get pretty expensive especially in multi-master, as each location has to not only store & maintain its own data but also the data (being sent) of everyone it sends to.
This replicate would also place each source document's revision ID in the the document's copy...
...that is ideally, including essential if the copy was to be {updated, aka a master}, too.
...in form of either:
ideally the normal "_rev" property. Indeed this looks quite possible per it ("preserve their revisions ID") already done by the normal replication algorithm using the builtin "Bulk Docs API" which seemingly our varient would use, too
otherwise have a new copy object (with its own _rev) plus another field as "_rev_original" ntelling the original rev. But well that would work?
Clearly such copy could be created no problem.
Probably no big if the destination is just reading the data.
Seems hairy if the destination is also writing the data. As we'd now have to merge with these non-standard revisions. But doable.
Relevant to this (coding an a custom/improved replication (to do this apparently-missing functionality) ideally without altering Pouch and especially Couch source code), as starter/basis material (the standard method), here's the normal Couch replication algorithm which unfortunately doens't clearly say it only uses builtin ops but it looks like it, and also the official overview of what it does; I'm suspecting Pouch implements this, likely in Pouch's replicate.js (latest release as of 2014.07).
Futher implementation particulars? - those who would know, please put it here.
This is a "community wiki" answer so please extend it.
Also please comment links & details of anyone/system already doing or trying to do this or similar.
I have certain objects in my domain which are not aggregate roots/entities, yet I still need to retrieve them from a database. I don't want to confuse things by creating repositories for these things. So, what are alternative data access patterns? Would you simply create a DAO for them, while still of course separating the interface?
Edit:
Some more detail on what I'm doing. I need to create a code. This code has certain rules as to its format. One of the rules is that the final character must be a unique number incremented by one from the last code generated. For example:
ABCD1
ABCD2
ABCD3
So, I'm keeping a table with one row, one column to store the number in question. Now, I don't want to consider this number an entity and create a repository for it - that's overkill. I just need a way of retrieving the number, adding 1 to it, and saving it. I know there are myriad ways I could do it, but I'm wondering if there's an customary way.
There are several data access patterns that could apply, in theory. You'd need to provide more detail though if you want us to suggest a specific pattern.
Without more detail, all I can suggest is to consider looking into Martin Fowler's Patterns of Enterprise Application Architecture book.
Edit: Customary way? No, not that I can think of - it really depends on where and how you're using this unique code in your domain. If I were doing this, I'd probably create a small service that speaks directly to the database to perform this function - not as heavy-weight as a repository, and very focused on the problem at hand.
Based on the edit: I would look first at the context in which you need to create that code. Perhaps there are some related entities or something that you are missing.
btw, I find the question really interesting as it comes up from time to time while coding specific features. I usually end up finding I was missing something on the scenario and it ends up fitting well with the normal repository pattern.
After surveying the options I'm going with the Table Gateway pattern.