I'm about to do an ExpressionEngine v1 to ExpressionEngine v2 Upgrade with lots of data stored in nGen File Fields.
What are the steps I need to take pre and post upgrade to get this data working correctly with the EE2 SafeCracker File field?
After upgrading to EE2, find each ex-nGen File field and change its Field Type to File, and run this SQL query:
UPDATE exp_channel_data
SET field_id_X = CONCAT('{filedir_Y}', field_id_X)
WHERE field_id_X != ''
AND field_id_X NOT LIKE '{filedir_%'
Replace “X” with your File field’s ID (you can get that from exp_channel_fields), and Y with the upload preference ID that nGen File Field was set to.
If you had Matrix installed in EE1, upgrade to Matrix 2/EE2 and do the same for any ex-nGen File columns, using this SQL query instead:
UPDATE exp_matrix_data
SET col_id_X = CONCAT('{filedir_Y}', col_id_X)
WHERE col_id_X != ''
AND col_id_X NOT LIKE '{filedir_%'
Again, X == your Matrix column ID (you can get that from exp_matrix_cols), and Y == your upload preference ID.
(Credit goes to Rob Sanchez, of course.)
I wrote this for a site as well - no matrix support in here but it works quickly and correctly for regular fields.
For the array, the left column is each field ID that you changed to text pre-upgrade, and needs to be changed to file post upgrade, and the right side is the X of filedir_X for the file upload dir you want to attach to the field
// Insert file upload directories
$array = array(
'16' => '1',
'22' => '1',
'121' => '3',
'58' => '1',
'67' => '1',
'68' => '1',
'71' => '1',
'76' => '1',
'78' => '1',
'94' => '1',
'99' => '1',
'108' => '3',
'109' => '3',
'110' => '3',
'139' => '1'
);
foreach($array as $field_id => $dir_id) {
$q_entries = $this->EE->db->query("SELECT entry_id, field_id_{$field_id} as 'field' from exp_channel_data where field_id_{$field_id} != '' order by entry_id asc");
if ($q_entries->num_rows() > 0) {
echo '<h3>field_id_'.$field_id.'</h3>';
foreach($q_entries->result_array() as $entry) {
echo $entry['entry_id'];
$filename = trim('{filedir_'.$dir_id.'}'.$entry['field']);
echo ' - '.$filename.'<br/>';
$data = array(
'field_id_'.$field_id => $filename,
);
$sql = $this->EE->db->update_string('exp_channel_data', $data, "entry_id = '{$entry['entry_id']}'");
$this->EE->db->query($sql);
}
}
}
echo 'done';
I have an entire blog post about this which is based off of my experience and the thread that Brandon referenced in his answer. Blog post here.
I've posted something to GitHub that should help anyone facing this process. It's EE1 template code (it needs PHP enabled on either output or input) that shows two things:
Firstly, it displays a tabular overview of all custom fields in the system. This is for reference, to help when you're trying to find all instances of a certain fieldtype. It covers normal EE fields, Fieldframe fieldtypes, and even Matrix columns.
Secondly, each time an nGen File field is encountered, the template generates the MySQL code you'll need to use (after you upgrade to EE2) to alter the data in said fields to the format required by EE2's native File field. These queries are only displayed, not run. The idea is that you save the queries somewhere, run the EE1->EE2 upgrade, and then run the saved queries when ready.
Needless to say, backup, Backup, BACKUP. The template code does not modify your site database in any way, and has been tested and displays what its supposed to just fine. However, while the MySQL queries it generates (for you to copy and manually run later) match what Brandon recommended in his answer, I have yet to actually test those queries in action.
The best way to do this is to proceed carefully and make backups of your database at each step of the way. I have previously written a blog post on this but will elaborate further.
Step 1: Before running your upgrade change all ngen field types to text, don't worry the data won't be lost.
Step 2: Next upgrade ExpressionEngine as per the official docs and then go back into each field and change them to the first party file type.
The next step involves a little bit of database manipulation, but it's just copying and paste so don't worry.
Step 3: Do make a back up of your database before you proceed just in case.
Step 4: This next step depends whether your original nGen file field was in a standard Channel Field or a Matrix Field.
Now go into your database and Replace “X” with your File field’s ID (you can get that from exp_channel_fields), and Y with the upload preference ID that nGen File Field was set to.
(To find your upload preference ID in your control panel, go to Content > Files > File Upload Preferences. Choose the ID column on the left that matches the file upload location.)
4a: If updating standard Channel Fields, use this query
UPDATE exp_channel_data
SET field_id_X = CONCAT('{filedir_Y}', field_id_X)
WHERE field_id_X != ''
AND field_id_X NOT LIKE '{filedir_%'
4b: For matrix, fields run this query instead
UPDATE exp_matrix_data
SET col_id_X = CONCAT('{filedir_Y}', col_id_X)
WHERE col_id_X != ''
AND col_id_X NOT LIKE '{filedir_%'
X == your Matrix column ID (you can get that from exp_matrix_cols), and Y == your upload preference ID.
Credit to Brandon Kelly and Rob Sanchez.
Additionally, the same procedure can be used for other add-ons that don't exist in EE2. Convert to text before the upgrade and then convert to a new equivalent field type post upgrade if needed. For more help: Click here
Related
I am currently building some kind of video channel based on an extension I created. It consists of videos and playlist that contains videos (obviously).
I have to create a page which contains a list of videos AND playlist by category. You also can sort those items by date. Finally, the page is paginated with an infinite scrolling that should load items 21 by 21.
To do so, I created on both Video and Playlist repositories a "findByCategory" function which is really simple :
$query = $this->createQuery();
return $query->matching($query->equals('categorie.uid',$categoryUid))->execute()->toArray();
Once I requested the items I need, I merge them in one array and do my sorting stuff. Here is my controller show action :
if ($this->request->hasArgument('sort'))
$sort = $this->request->getArgument('sort');
else
$sort = 'antechrono';
//Get videos in repositories
$videos = $this->videoRepository->findByCategorie($categorie->getUid());
$playlists = $this->playlistRepository->findByCategorie($categorie->getUid());
//Merging arrays then sort it
if ($videos && $playlists)
$result = array_merge($videos, $playlists);
else if ($videos)
$result = $videos;
else if ($playlists)
$result = $playlists;
if ($sort == "chrono")
usort($result, array($this, "sortChrono"));
else if ($sort == "antechrono" || $sort == null)
{
usort($result, array($this, "sortAnteChrono"));
$sort="antechrono";
}
$this->view->assignMultiple(array('categorie' => $categorie, 'list' => $result, 'sort' => $sort));
Here is my view :
<f:widget.paginate objects="{list}" as="paginatedList" configuration="{addQueryString: 'true', addQueryStringMethod: 'GET,POST', itemsPerPage: 21}">
<div class="videos row">
<f:for each="{paginatedList}" as="element">
<f:render partial="Show/ItemCat" arguments="{item: element}"/>
</f:for>
</div>
</f:widget.paginate>
The partial render shows stuff including a picture used as a cover. So I need at least this relation in the view.
This works fine and shows only the items from the category that is requested. Unfortunatly I have a huge performance issue : I tried to show a category that contains more than 3000 records and It takes about one minute to load. It's a little bit long.
By f:debugging my list variable, I see that it contains every records even through it shouldn't be the case (that's the point of pagination...). So the first question is : is there something wrong in the way I did my pagination ?
I tried to simplify my requests by enabling the rawQuery thing ($query->execute(true)) : I get way better performance, but I can't get the link for the pictures (in my view, I get 1 or 0 but not the picture's uid...). Second question : is there a way to fix this issue ?
I hope my description is clear enough. Thanks for your help :-)
When you execute a query, it will not actually fetch the data from the database until the results are accessed. If the paginate widget gets a query result it will add limits and offset to the query and then fetch the data from the database, so you will only get the records that are shown on a page.
In your case you added toArray() after execute(), which accesses the results, so the data is fetched from the database and you get all records. The best solution I can think of is to combine the 2 tables into 1 so you can do it with a single query and don't have to merge and order them in PHP.
As long as you sort the data after the query you have to handle all data (request all records and especially resolve all relations).
Try to sort the data in the query itself (order by), so you could restrict the data to only those records which are needed for the current 'page' (limit <offset>,<number>).
Here a complex query with join and limit could be faster than a full query and filtering in PHP.
I have a site that uses Wayfinder to display the latest 3 entries from an Articles blog. Now, I want to only consider those blog entries that are tagged Highlights.
My original Wayfinder call looks like this, nothing spectacular:
[[!Wayfinder? &startId=`296` &level=`1`
&outerTpl=`emptyTpl`
&innerTpl=``
&rowTpl=`thumbnails_formatter`
&ignoreHidden=`1`
&sortBy=`menuindex`
&sortOrder=`DESC`
&limit=`3`
&cacheResults=`0`
]]
as Articles tags are managed via the articlestags TV, I thought that a &where might do the trick, but with no luck yet:
&where=`[{"articlestags:LIKE":"%Highlights%"}]`
does not yield anything. As a sanity check, I tried [{"pagetitle:LIKE":"%something%"}], which worked. Obviously, the problem is that articlestags is not a column of modx_site_content, but I'm not sure about how to put the subquery.
SELECT contentid
FROM modx_site_tmplvar_contentvalues
WHERE tmplvarid=17
AND value LIKE '%Highlights%'
Gave me the right IDs on the sql prompt, but adding it to the Wayfinder call like this gave an empty result again:
&where=`["id IN (SELECT contentid FROM modx_site_tmplvar_contentvalues WHERE tmplvarid=17 AND value LIKE '%Highlights%')"]`
Any ideas on how to achieve this? I'd like to stay with Wayfinder for consistency, but other solutions are welcome as well.
You can just use pdomenu (part of pdoTools) instead Wayfinder
[[!PdoMenu?
&startId=`296`
&level=`1`
&outerTpl=`emptyTpl`
&innerTpl=``
&rowTpl=`thumbnails_formatter`
&ignoreHidden=`1`
&sortBy=`menuindex`
&sortOrder=`DESC`
&limit=`3`
&cacheResults=`0`
&includeTVs=`articlestags`
&where=`[{"TVarticlestags.value:LIKE":"%filter%"}]`
]]
Take a peek at some of the config files [core/components/wayfinder/configs ] - I have not tried it, but it looks as if you can run your select query right in the config & pass the tmplvarid array to the $where variable.
A little playing around led me to a solution: I needed to include the class name (not table name) when referring to the ID:
&where=`["modResource.id IN (SELECT contentid FROM modx_site_tmplvar_contentvalues WHERE tmplvarid=17 AND value LIKE '%Highlights%')"]`
a small test showed that even a simple
&where=`["id = 123"]`
does not work without modResource..
A look at wayfinder.class.php shows the following line, which seems to be the "culprit":
$c->select($this->modx->getSelectColumns('modResource','modResource'));
This method aliases the selected columns - relevant code is in xpdoobject.class.php. The first parameter is the class name, the second a table alias. The effect is that the query selects id AS modResource.id, and so on.
EDIT: final version of my query:
&where=`["modResource.id IN (
SELECT val.contentid
FROM modx_site_tmplvars AS tv
JOIN modx_site_tmplvar_contentvalues AS val
ON tv.id = val.tmplvarid
WHERE tv.name = 'articlestags' AND (
val.value = 'Highlights'
OR val.value LIKE 'Highlights,%'
OR val.value LIKE '%,Highlights'
OR val.value LIKE '%,Highlights,%'
)
)"]`
I don't claim this query is particularly efficient (I seem to recall that OR conditions are bad). Also, MODx won't work with this one if the newlines aren't stripped out. Still, I prefer to publish the query in its well-formatted form.
I used snippet as a parameter for the includeDocs of wayfinder, In my case it was useful because I was need different resources in menu depend on user browser (mobile or desktop)
[[!Wayfinder?
&startId=`4`
&level=`1`
&includeDocs=`[[!menu_docs?&startId=`4`]]`
&outerTpl=`home_menu_outer`
&rowTpl=`menu_row`
]]
and then menu_docs snippet
<?php
if (empty ($startId))
return;
if (!isMobileDevice())
return;
$query = $modx->newQuery('modResource');
$query->innerJoin('modTemplateVarResource','TemplateVarResources');
$query->where(array(
'TemplateVarResources.tmplvarid' => 3,
'TemplateVarResources.value:LIKE' => 'yes',
'modResource.parent' => $startId,
'modResource.deleted' => 0,
'modResource.published' => 1,
'modResource.hidemenu' => 0
));
$resources = $modx->getCollection('modResource', $query);
$ouput = array();
foreach ($resources as $resource)
$output[] = $resource->get('id');
return implode (',', $output);
I seem to be having a problem with assigning values to fields of a content item with a custom content part and the values not persisting.
I have to create the content item (OrchardServices.ContentManager.Create) first before calling the following code which modifies a field value:
var fields = contentItem.As<MyPart>().Fields;
var imageField = fields.FirstOrDefault(o => o.Name.Equals("Image"));
if (imageField != null)
{
((MediaLibraryPickerField)imageField).Ids = new int[] { imageId };
}
The above code works perfectly when against an item that already exists, but the imageId value is lost if this is done before creating it.
Please note, this is not exclusive to MediaLibraryPickerFields.
I noticed that other people have reported this aswell:
https://orchard.codeplex.com/workitem/18412
Is it simply the case that an item must be created prior to amending it's value field?
This would be a shame, as I'm assigning this fields as part of a large import process and would inhibit performance to create it and then modify the item only to update it again.
As the comments on this issue explain, you do need to call Create. I'm not sure I understand why you think that is an issue however.
The following code is a script object on an XPage in it I loop through an array of all the forms in a database, looking for all the forms that contain the field "ACIncludeForm". My method works but it takes 2 - 3 seconds to compute which really slows the load of the XPage. My question is - is there a better method to accomplish this. I added code to check to see if the sessionScope variable is null and only execute if needed and the second time the page loads it does so in under a second. So my method really consumes a lot of processor time.
var forms:Array = database.getForms();
var rtn = new Array;
for (i=0 ; i<forms.length; ++i){
var thisForm:NotesForm = forms[i];
var a = thisForm.getFields().indexOf("ACIncludeForm");
if (a >= 0){
if (!thisForm.isSubForm()) {
if (thisForm.getAliases()[0] == ""){
rtn.push(thisForm.getName() + "|" + thisForm.getName() );
}else{
rtn.push(thisForm.getName() + "|" + thisForm.getAliases()[0] );
}
}
}
thisForm.recycle()
}
sessionScope.put("ssAllFormNames",rtn)
One approach would be to build an index of forms by yourself. For example, create an agent (LotusScript or Java) that gets all forms and for each form, create a document with for example a field "form" containing the form name and and a field "fields" containing all field names (beware of 32K limit).
Then create a view that displays all these documents and contains the value of the "fields" field in the first column so that each value of this field creates one line in this view.
Having such a view, you can simply make a #DbLookup from your XPage.
If your forms are changed, you only need to re-run the agent to re-build your index. The #DbLookup should be pretty fast.
Place the form list in a static field of a Java class. It will stay there for a long time (maybe until http boot). In my experience applicationScope values dissappear in 15 minutes.
I am making some changes to a site using ExpressionEngine, there is a concept I can't seem to quite understand and am probably coding workarounds that have pre-provided methods.
Specifically with channels and members, the original creator has added several custom fields. I am from a traditional database background where each column in a table has a specific meaningful name. I am also used to extending proprietary data by adding a related table joined with a unique key, again, field names in the related table are meaningful.
However, in EE, when you add custom fields, it creates them in the table as field_id_x and puts an entry into another table telling you what these are.
Now this is all nice from a UI point of view for giving power to an administrator but this is a total headache when writing any code.
Ok, there's template tags but I tend not to use them and they are no good in a database query anyway.
Is there a simple way to do a query on say the members table and then address m_field_1 as what its really called - in my case "addresslonglat".
There are dozens of these fields in the table I am working on and at the moment I am addressing them with fixed names like "m_field_id_73" which means nothing.
Anybody know of an easy way to bring the data and its field names together easily?
Ideally i'd like to do the following:
$result = $this->EE->db->query("select * from exp_member_data where member_id = 123")->row();
echo $result->addresslonglat;
Rather than
echo $result->m_field_id_73;
This should work for you:
<?php
$fields = $data = array();
$member_fields = $this->EE->db->query("SELECT m_field_id, m_field_name FROM exp_member_fields");
foreach($member_fields->result_array() as $row)
{
$fields['m_field_id_'.$row['m_field_id']] = $row['m_field_name'];
}
$member_data = $this->EE->db->query("SELECT * FROM exp_member_data WHERE member_id = 1")->row();
foreach($member_data as $k => $v)
{
if($k != 'member_id')
{
$data[$fields[$k]] = $v;
}
}
print_r($data);
?>
Then just use your new $data array.