Flask-Sqlalchemy data mutation when replacing text - python-3.x

I'm using flask-sqlalchemy on a site to save some information from a textarea. This text area could contain some simple bbcode that I would handle later to allow text formatting.
When I save or update anything, everything looks normal before and after the update operation. When I go to edit that same information, it has <br>, <strong> and other tags that have been placed in the text. Is this something that flask-sqlalchemy does? If so, how do I disable this? It seems like the problem is in flask-sqlchemy based on where the problem happens, and I'd rather handle any formatting myself.
I've also checked how the data is stored in the database. The database contains the HTML code rather than the bbcode markup, so the problem does not lie in the templates that display the editing view.
EXAMPLE
In the form, I submit this data.
[b]hello world[/b]
how are you
In the flask route for the update, I print out the data
#app.route("/update/item/<id>", methods=['POST']
def update(id):
# While testing I'm not going to bother with escaping anything.
# Once I figure out what's causing this, I'll do so with the escape() function.
item = request.form['item']
print(repr(item)) # "[b]Hello World[/b]\r\n\r\nhow are you"
Item.query.filter_by(id=id).update(dict(item=str(item))
queried = Item.query.filter_by(id=id).first()
print(queried.item) # "[b]Hello World[/b]\r\n\r\nhow are you"
When i go to edit the item, it displays this in the text area
<strong>Hello World</strong>
<br>
<br>how are you
rather than
[b]Hello World[/b]
how are you
UPDATE:
This only appears to happen when the following code is ran
#...
def paginate(page):
items = Item.query.order_by(Item.id.desc()).paginate(page=page, per_page=10)
for text in items:
# Just replacing [b] and [/b] as an example here. I'll have everything
# regexed out and replaced in a dedicated function.
# This replacing appears to be the culprit. Commenting this out fixes it.
text.item = text.item.replace("[b]", "\<strong\>")
text.item = text.item.replace("[/b]", "\</strong\>")
return items

This problem was brought about by not knowing that sqlalchemy automatically saves changes back to the database when an attribute is changed, for example, by parsing the bbcode for later use. While I may have strong opinions on mutable data structures automatically saving themselves (it should NEVER happen), here's how to fix this if anyone else has the same problem.
First, know that sqlalchemy will automatically save if you modify anything on your sql query unless you detach it from the database, EVEN IF you use autoflush=False and autocommit=False (I tried both). Before making any changes, call the expunge() function and then make your changes. This detaches the query from the database and allows you to modify to your heart's content without any (potentially devastating) data mutation happening in your database.
In my case, this means my paginate function should look like this
# Make sure you have your db object imported somewhere above
def paginate(page):
items = Item.query.order_by(Item.id.desc()).paginate(page=page, per_page=10)
for text in items:
db.session.expunge(text)
text.item = text.item.replace("[b]", "<strong>")
text.item = text.item.replace("[/b]", "</strong>")
return items

Related

Why am I getting duplicate lines when inserting and updating a new row?

I am generating a production order and inserting material lines. When creating them, I have to set some extension fields. To get to those fields, I need to perform a graph.view.Insert(row) action. After, I can access the extended fields. After setting all of the extended fields, I need to perform a graph.view.Update(row) so that those values are set in the cache. After I am done, I perform a graph.Actions.PressSave(). When I execute this and navigate to the details record, I see two lines for each detail instead of just one. I've noticed this happening on any grid I insert a record into, set values, and then perform an update before saving. Is there any reason for this?
Here is some example code of what I'm doing:
AMProdMatl newMat = new AMProdMatl();
//Set values
graph.ProdMatlRecords.Insert(newMat);
AMProdMatlExt newMatExt = newMat.GetExtension<AMProdMatlExt>();
//Set extended values
graph.ProdMatlRecords.Update(newMat);
graph.Actions.PressSave();
The result:
I want to emphasize again that I've run into this issue in other sections of Acumatica and I don't think this is a Production module exclusive issue.
Make sure you are using the return of the insert and update statements before reusing the row in additional statements.
For Example:
newMat = graph.ProdMatlRecords.Insert(newMat);

Sql to Rename Orchard Custom Content Type

I'm planning to rename one of my custom content types, so that I can free up its name for a new Orchard module I am working on. I'm hoping to use Schema.ExecuteSql in a migration class as suggested by this SO answer, but I want to make sure I know all of the updates I will need to do.
So far, I understand that I need to update fields in the following tables:
Orchard_Framework_ContentTypeRecord
Settings_ContentTypeDefinitionRecord
Settings_ContentPartDefinitionRecord
Also, here is my general plan for the update SQL I will need to run:
DECLARE #From VARCHAR(50) = 'OriginalName'
DECLARE #To VARCHAR(50) = 'NewName'
BEGIN TRANSACTION
BEGIN TRY
UPDATE [Current_Orchard_Framework_ContentTypeRecord]
SET [Name] = #To
WHERE [Name] = #From
UPDATE [Current_Settings_ContentTypeDefinitionRecord]
SET [Name] = #To, [DisplayName] = #To
WHERE [Name] = #From
UPDATE [dbo].[Current_Settings_ContentPartDefinitionRecord]
SET [Name] = #To + 'Part'
WHERE [Name] = #From + 'Part'
--COMMIT TRANSACTION
ROLLBACK TRANSACTION /*Rollback while testing*/
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION
END CATCH
Is there anything else I am missing that will need to be renamed to fully rename my content type?
This might not be the exact answer to your problem, but - although it has annecdotal character - it might help you find the solution.
I was working on a Orchard site that had
a lot of pages
a lot of html links to other pages on the site
a lot of custom link items in the navigations (to link to certain anchors inside a target page, etc.)
Shortly before we planned to delpoy the website, it was decided to change the URLs of the most prominent pages for SEO reasons from http://www.example.org/orchard/products/category-name/product-nameto http://www.example.org/orchard/shop/category-name/product-name-keyword-anotherkeyword(gotta love SEO...)
Doing this manually would have taken ages. Changing the URL in the content item itself would be easy (but doing that on ~20 items still would take some time), but changing all those <a hrefs in all the 120+ content items, and all the custom links in the 5 navigations (5 languages) would simply be impossible.
So the only feasable way seemed to be database manipulation.
I used MS SQL Server Management Studio to export all tables in the Orchard database to individual .sql files. I then used Notepad++ to search for all files that contained the partial string /products/category/product-name and wrote down the table names and the field name (usually Text or Data)
In this case the results were in the following tables:
Common_BodyPartRecord.Table
Orchard_Framework_ContentItemRecord
Orchard_Framework_ContentItemVersionRecord
To search-replace a partial string in SQl you use the replace function, and because of the datatype of the table fields the orchard stores the data in you have to cast strings. The resulting SQL queries look like this:
update [Orchard_Live].[dbo].[Common_BodyPartRecord]
set Text = cast(replace(cast(Text as nvarchar(max)), N'/products/category/product-name', N'/shop/category-name/product-name-keyword-anotherkeyword') as ntext)
where cast(Text as nvarchar(max)) LIKE N'%/products/category/product-name%'
update [Orchard_Live].[dbo].[Orchard_Framework_ContentItemRecord]
set Data = cast(replace(cast(Data as nvarchar(max)), N'/products/category/product-name', N'/shop/category-name/product-name-keyword-anotherkeyword') as ntext)
where cast(Data as nvarchar(max)) LIKE N'%/products/category/product-name%'
update [Orchard_Live].[dbo].[Orchard_Framework_ContentItemVersionRecord]
set Data = cast(replace(cast(Data as nvarchar(max)), N'/products/category/product-name', N'/shop/category-name/product-name-keyword-anotherkeyword') as ntext)
where cast(Data as nvarchar(max)) LIKE N'%/products/category/product-name%'
I had to do this for every URL that had to be changed. Took quite a while, but still beat manually changing everything by far. Maybe this "workflow" can help with your problem, too.

Retrieving column values in filtered xpages view

I have a view defined on an xpage. I also have several filters (based on the columns) that the user can select and combine to filter the results in the view. I generate a query string based on this that I construct in dominoView.search (doing a complete refresh). What I would like to do is get the results of the search so that I can then update some counts displayed elsewhere on the page. I'm having a hard time figuring out where I can perform this logic, though. I'm trying to use view.getAllEntries() and then iterating over the collection. Sometimes it seems like it works, but other times I seem to be getting the unfiltered view. Someone suggested I explicitly call view.FTSearch inside one of the events (beforePageLoad?) and immediately after do my getAllEntries call, saving the results in viewScope, but I get an "Error while browses Notes view" runtime error when I try to do that. Any pointers? TIA!
EDIT: After studying the xpages lifecycle a bit (which is still a little confusing), I think I can fine-tune my question. This is my first stackoverflow question, so I hope this is okay to do and productive....
As I described, I have a dominoView defined on my xpage. A repeat iterates over the rows of the view, displaying certain fields from the documents. If I define a query in the search property, then the repeat correctly displays the reduced set of documents rather than the complete set. (The query is computed in the search property via SSJS from some variables defined in the viewScope in a combobox's eventHandler.) However, if I try to access the current entries in the view inside of the repeat's rendered section (with SSJS) using myView.getAllEntries (where myView is what's defined as the "value" of the repeat), I am still getting all of the documents, even if a query has been done. It seems like at that point, the view variable has already had its search applied (since the repeat works), so why the differing results? Is there another way to access the view's rows? To complicate this further, this is just a simple experiment that might clarify the problem; as I indicated earlier, I don't actually want to access the view data within the repeat, I want to access it in the rendered or value sections of some comboboxes defined before the repeat in the xpage file.
I hope that makes more sense now....
EDIT #2: I forgot to add that if I manually call FTSearch (or FTSearchSorted) before calling myView.getAllEntries, then I think I can make this work. It just seems unnecessary to have to do that in addition to the view's built-in search.
From what I get you want to iterate over the entries in a view that before has been filtered, i.e. whose resulting entry collection is smaller than the the view itself.
What I don't get (yet) is what you want to do with the result, or what you're axpecting to get from the iteration over your filtered view (you're mentioning some counts to be displayed somewhere else).
Probably a good way is to use the view's .getAllEntriesByKey method which returns a NotesViewEntryCollection object which then can be used for your iteration.
Don't forget to recycle the resulting NotesViewEntry objects; reason for this has been explained several times here at stackoverflow.

Storing index values in FAST-ESP without modifications

first of all I'm totally new to FAST but I already have a couple of issues I need to solve, so I'm sorry if my questions are very basic =)
Well, the problem is that I have a field in the FAST index which in the source document is something like "ABC 12345" (please note the intentional whitespaces) but when stored in the index is in the form "ABC 123456" (please note that now there is a single space).
If I retrieve all the document values then this specific value is OK (with all the whitespaces), my only problem is with the way the value is stored in the index since I need to retrieve and display it to my user just like it appears in the original document, and I don't want to go to the full document just for this value, I want the value that I already have in the index. I think I need to update one of the FAST XML configuration files but I don't have enough documentation at hand in order to decide where to perform the change, index_profile.xml? in the XMLMapper file?
I've found the answer by myself. I'm using a XMLMapper for my collection, all I had to do was to add the ignore-whitespace attribute to the Mapping element and then set this attribute value to "false". This solved the problem and the raw data now when retrieved from the index contains the expected inner whitespaces.
Thanks.

Help to restructure my Doc/View more correctly

Edited by OP.
My program is in need of a lot of cleanup and restructuring.
In another post I asked about leaving the MFC DocView framework and going to the WinProc & Message Loop way (what is that called for short?). Well at present I am thinking that I should clean up what I have in Doc View and perhaps later convert to non-MFC it that even makes sense. My Document class currently has almost nothing useful in it.
I think a place to start is the InitInstance() function (posted below).
In this part:
POSITION pos=pDocTemplate->GetFirstDocPosition();
CLCWDoc *pDoc=(CLCWDoc *)pDocTemplate->GetNextDoc(pos);
ASSERT_VALID(pDoc);
POSITION vpos=pDoc->GetFirstViewPosition();
CChildView *pCV=(CChildView *)pDoc->GetNextView(vpos);
This seem strange to me. I only have one doc and one view. I feel like I am going about it backwards with GetNextDoc() and GetNextView(). To try to use a silly analogy; it's like I have a book in my hand but I have to look up in it's index to find out what page the Title of the book is on. I'm tired of feeling embarrassed about my code. I either need correction or reassurance, or both. :)
Also, all the miscellaneous items are in no particular order. I would like to rearrange them into an order that may be more standard, structured or straightforward.
ALL suggestions welcome!
BOOL CLCWApp::InitInstance()
{
InitCommonControls();
if(!AfxOleInit())
return FALSE;
// Initialize the Toolbar dll. (Toolbar code by Nikolay Denisov.)
InitGuiLibDLL(); // NOTE: insert GuiLib.dll into the resource chain
SetRegistryKey(_T("Real Name Removed"));
// Register document templates
CSingleDocTemplate* pDocTemplate;
pDocTemplate = new CSingleDocTemplate(
IDR_MAINFRAME,
RUNTIME_CLASS(CLCWDoc),
RUNTIME_CLASS(CMainFrame),
RUNTIME_CLASS(CChildView));
AddDocTemplate(pDocTemplate);
// Parse command line for standard shell commands, DDE, file open
CCmdLineInfo cmdInfo;
ParseCommandLine(cmdInfo);
// Dispatch commands specified on the command line
// The window frame appears on the screen in here.
if (!ProcessShellCommand(cmdInfo))
{
AfxMessageBox("Failure processing Command Line");
return FALSE;
}
POSITION pos=pDocTemplate->GetFirstDocPosition();
CLCWDoc *pDoc=(CLCWDoc *)pDocTemplate->GetNextDoc(pos);
ASSERT_VALID(pDoc);
POSITION vpos=pDoc->GetFirstViewPosition();
CChildView *pCV=(CChildView *)pDoc->GetNextView(vpos);
if(!cmdInfo.m_Fn1.IsEmpty() && !cmdInfo.m_Fn2.IsEmpty())
{
pCV->OpenF1(cmdInfo.m_Fn1);
pCV->OpenF2(cmdInfo.m_Fn2);
pCV->DoCompare(); // Sends a paint message when complete
}
// enable file manager drag/drop and DDE Execute open
m_pMainWnd->DragAcceptFiles(TRUE);
m_pMainWnd->ShowWindow(SW_SHOWNORMAL);
m_pMainWnd->UpdateWindow(); // paints the window background
pCV->bDoSize=true; //Prevent a dozen useless size calculations
return TRUE;
}
Thanks
Hard to give you good recommendations without knowing what your program shall do. I have only a few general remarks:
Your InitInstance does not look very messed up for me. It's pretty much standard with a bit of custom code in it.
Also the ugly construction to retrieve the first view from the application class (the chain GetDocTemplate -> GetDoc -> GetView) is standard to my knowledge. I actually don't know another way. You might think about moving it into a separate method like CChildView* CLCWApp::GetFirstView() but well, that's only cosmetic as long as you need it only at one place.
What you are doing and which data you are placing in your Document class and in your View class(es) is more a semantic question if you only have one view. (You have only one document anyway because it's an SDI application.). From a technical viewpoint often both is possible.
But to be open for (perhaps) later extensions to more than one view and to follow the standard pattern of a doc/view architecture there are a few rules of thumb:
Data which exist and have a meaning independent of the way to present and view them (a document file, a database handle, etc.) belong to the document class. I don't know what your pCV->OpenF1(cmdInfo.m_Fn1) ... and so on does but if it's something like a file or filename or a parameter to be used to access data in any way OpenF1 might be better a method of the document class.
Methods which do any kind of data processing or modification of your underlying data belong to the document class as well
Data and methods which are only needed for a specific way to display a document belong to a view class (for instance a selected font, colours, etc.)
On the other side: If you have a fixed number of views which open with the document it might not be wrong to put view specific data into the document, especially if you want to make those view parameters persistent. An example would be a file with some statistical data - your document - and a splitter frame with two views: one displays the data as a grid table and the other as a pie chart. The table has "view data" describing the order of and width of columns, the pie chart has data to configure the colours of the pie pieces and the legend location, for instance. If you want to make sure that the user gets the last view configuration displayed when he opens the document file you have to store these view parameters somewhere. It wouldn't be wrong or bad design in my opinion to store those parameters in the document too, to store and retrieve them from any permanent storage, even if you need them only in the view classes.
If your application allows to open an unlimited number of views for a document dynamically and those views are only temporary as long as the application runs, storing all view configuration parameters directly in the view classes seems more natural to me. Otherwise in the document you would need to manage any kind of dynamic data structure and establish a relationship between a View and an entry in this data structure (an index in an array, or a key in a map, etc.)
If you are in doubt whether to place any data in the document or view class I'd prefer the document because you always have the easy GetDocument() accessor in the View class to retrieve members or call methods of the Doc. To fetch data from the View into the Document requires to iterate through the list of views. (Remember: Doc-View is a 1-n relationship, even in a SDI application.)
Just a few cents.

Resources