When is ActiveStorage::IntegrityError raised? - rails-activestorage

My app (locally) raises ActiveStorage::IntegrityError error, whenever it tries to attach a file. How can I get out of this error?
I have only one has_one_attached and I don't know how that error gets in the way.
# model
has_one_attached :it_file
Tempfile.open do |temp_file|
# ...
it_file.attach(io: temp_file, filename: 'filename.csv', content_type: 'text/csv')
end
# storage.yml
local:
service: Disk
root: <%= Rails.root.join("storage") %>
EDIT: it can be related with deleting storage/ directory (it happened after I deleted that) or it can be because it's happening in a job (the full error was Error performing ActivityJob (Job ID: .. ) from Async( .. ) in .. ms: ActiveStorage::IntegrityError (ActiveStorage::IntegrityError)
And this does not add files to storage/ folder but it's generating folders under it when I tried to attach them.

As mentioned in the comments, one reason this can happen is that the file object is at the end of the file, which was the problem in this case. It could be fixed here with temp_file.rewind.

Very weird. After update to rails 6.0 I must recalculate some checksums. Yes I used dokku,docker. It was fine before update.
# Disk service is in use for ActiveStorage
class ProjectImage < ApplicationRecord
has_one_attached :attachment
end
# update all checksums
ProjectImage.all.each do |image|
blob = image.attachment.blob
blob.update_column(:checksum, Digest::MD5.base64digest(File.read(blob.service.path_for(blob.key))))
end;

This was happening to me not because of anything mentioned above.
In my case I was defining a test var using one dummy file, but I was attaching it to 2 different records.
let(:file) { File.open(Rails.root.join('spec', 'fixtures', 'files', 'en.yml')) }
let(:data) { [file, file] }
The function in question received a list of ids and data and attaching the files to the records. This is a simplified version of the code
record_0.file.attach(
io: data[0],
filename: 'en.yml',
content_type: 'application/x-yaml'
)
record_1.file.attach(
io: data[1],
filename: 'en.yml',
content_type: 'application/x-yaml'
)
Once I defined 2 test vars, one for each record, using the same file I got it to work.
let(:file_0) { File.open(Rails.root.join('spec', 'fixtures', 'files', 'en.yml')) }
let(:file_1) { File.open(Rails.root.join('spec', 'fixtures', 'files', 'en.yml')) }
let(:data) { [file_0, file_1] }

Background
In my case I faced this error when I was trying to upgrade the rails config defaults.
I was activating configs from config/initializers/new_framework_defaults_6_1.rb which was generated by rails app:update rake task.
Cause
I activated this setting
Rails.application.config.active_storage.track_variants = true
which collided with our existing mechanism to handle variant generation. We are reading variants size/types from account-settings so its complicated.
Technical cause
As mentioned above, this is causes due to mismatch in checksum of the file and checksum stored in blob record in database.
# activestorage-6.1.7.1/lib/active_storage/downloader.rb:37
def verify_integrity_of(file, checksum:)
unless Digest::MD5.file(file).base64digest == checksum
raise ActiveStorage::IntegrityError
end
end
Solution
I commented it back
To make sure its always inactive, I moved the setting to config/initializers/active_storage.rb like below
# Track Active Storage variants in the database.
# *Note*: Its Rails 6.1.0 feature and we have our own way of handling variants which also depends upon the
# thumbnail_sizes setting values and varies from account to account.
# so we are disabling it for now.
Rails.application.config.active_storage.track_variants = false
Summary
You may want to use this awesome feature so see workarounds.
In your case, causes can be something else so look deeper.
disabling this feature solves my issue.

Related

Error http 422 with DirectUpload / ActiveStorage / Rails

I try to add some images with ajax via DirectUpload / ActiveStorage / Rails 6.
I use the prerequisites into of ActiveStorage support, for use DirectUpload with Jquery :
https://edgeguides.rubyonrails.org/active_storage_overview.html#integrating-with-libraries-or-frameworks
const upload = new DirectUpload(file, url)
upload.create((error, blob) => {
if (error) {
// Handle the error
} else {
// Add an appropriately-named hidden input to the form with a
[..]
console.log(blob.key);
}
})
On my host, it works for all files. But when I try to publish my app into my hoster, I have an error for some files, always the same, after the request of DirectUpload :
Completed 422 Unprocessable Entity in 2ms (ActiveRecord: 0.0ms | Allocations: 689)
I looked the XHR requests into my webtools browser, but the payload seems the same into a file which works and another which fails :
{id: 219, key: "v2v1aqlk8gyygcc4smjeh0bbuc59", filename: "groupama logo.jpeg",…}
id: 219
key: "v2v1aqlk8gyygcc4smjeh0bbuc59"
filename: "logo.jpeg"
content_type: "image/jpeg"
metadata: {}
byte_size: 17805
checksum: "3GIVi2kNKClfH+d9HGYOfkA=="
created_at: "2020-04-09T08:25:40.000+02:00"
signed_id: "eyJfcmFpbHMiOnsibWVzc2zaFnZSI6IkJBaHBBZHM9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--7c0750cb8c86a955a04fa9a11dc5389cdeb5e7b0"
attachable_sgid: "BAh7CEkiCGdpZAY6BkVUSSIxsaZ2lkOi8vYXBwL0FjdGl2ZVN0b3JhZ2U6OkJsb2IvMjE5P2V4cGlyZXNfaW4GOwBUSSIMcHVycG9zZQY7AFRJIg9hdHRhY2hhYmxlBjsAVEkiD2V4cGlyZXNfYXQGOwBUMA==--64a945c38dc5d85c05156da50b9c38819b106e10"
direct_upload: {,…}
url: "http://localhost:8491/rails/active_storage/disk/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaDdDVG9JYTJWNVNTSWhkakoyTVdGeGJHczRaM2w1WjJOak5ITnRhbVZvTUdKaWRXTTFPUVk2QmtWVU9oRmpiMjaUwWlc1MFgzUjVjR1ZKSWc5cGJXRm5aUzlxY0dWbkJqc0dWRG9UWTI5dWRHVnVkRjlzWlc1bmRHaHBBbzFGT2cxamFHVmphM04xYlVraUhUTkhTVlpwTW10T1MwTnNaa2dyT1VoSFdVOW1hMEU5UFFZN0JsUT0iLCJleHAiOiIyMDIwLTA0LTA5VDA2OjMwOjQwLjg5NFoiLCJwdXIiOiJibG9iX3Rva2VuIn19--a2acedc0924f735c5cc08db8c4b76f76accc3c8d"
headers: {Content-Type: "image/jpeg"}
I tried this solution, by the monkey patch doesn't works for me, and another solution seems not working :
Rails API ActiveStorage DirectUpload produce 422 Error InvalidAuthenticityToken
I noticed, when I try to upload the logo image file without use DirectUpdate into input file, the file is correctly well send to my server.
= f.file_field :logos, direct_upload: true
Do you have any idea to test ?
My issue was coming with the IO which was use for copy the file. Into the ActiveStorage::DiskController#update, Rails use the request.body and IO.CopyStream for create the file, and a checksum file was done for verifiy the file created.
And the check fail and throw the 422 http error.
I noticed than the IO stream, in dev mode was a String_IO, whereas on my hoster, the IO was a Uswgi_IO. Because my hoster deliver the ruby on rails application with Uswgi.
The uwsgi_io not contains a length or size methods, and when ActiveStorage create the file with this IO, the size of file was weird. Weirdly too large.
I noticed if the RAW_POST_DATA was assign, then the request.body return a String_IO. And into the request.raw_post method, the body was read directly with the request.content_length :
raw_post_body.read(content_length)
https://api.rubyonrails.org/classes/ActionDispatch/Request.html#method-i-body
I created a new controller which inherits of ActiveStorage::DiskController, in order to assign the RAW_POST_DATA, before the Disk#update action.
class UploadController < ActiveStorage::DiskController
def update
request.env['RAW_POST_DATA'] = request.body.read(request.content_length)
super
end
end
And after, I override the ActiveStorage disk#update route by mine :
put '/rails/active_storage/disk/:encoded_token', to: 'upload#update
And it works !
ActionText works with ActiveStorage for store the images, and I had the same issue with the images too large.
My patch allows to make works ActionText on my hoster too.
Kudos for finding the culprit here, I was having the exact same issue on a uwsgi setup and I was trying to figure out why the content length was different when I found your post. Thanks for sharing !
I just took a slightly different approach with the fix as I didn't want to hack the AS routes, so I created a config/initializers file with the following code :
Rails.configuration.to_prepare do
ActiveStorage::DiskController.class_eval do
before_action :set_raw_post_data, only: :update
def set_raw_post_data
request.env['RAW_POST_DATA'] = request.body.read(request.content_length)
end
end
end
Maybe it would be worth creating an issue in https://github.com/unbit/uwsgi to let them know about this ?

Unable to determine function entry point

Been using MS botframework for a couple months. Developing with the emulator in node and using continuous integration to push to Azure.
Pushed last Wednesday and tested with no problems. Made two very inconsequential code changes and pushed on Friday and no I'm getting:
Exception while executing function: Functions.messages. mscorlib: Unable to determine function entry point. I tried redeploying the older version, same thing.
Thoughts?
The function entrypoint is determined based on this logic. As you can see, the flow is:
If an explicit entry point is defined in function.json, use that
Otherwise; if there's a single exported function, use that
Otherwise; try to use a function named run or index (in that order)
I suspect you were in branch #2, and your minor change introduced new functions, so the runtime is now attempting to locate a function named run or index and that doesn't exist.
Can you please make sure your primary entry point function is named run or index and try again?
Turns out there was a short-lived bug in the Azure git integration and I deployed during the window this bug was live. It modified function.json and left it in an invalid state. Kudos to MS Support for staying with the issue and determining the root cause.
In my case, the root cause is having 1 named export & 1 default export.
To fix this, only export default 1 thing in the entrypoint file (index.js in my case)
Leaving here as a trail in case someone faces the same thing
1) Try stopping the service in Azure:
2) Then go to Kudu: https://[YourAzureSiteName].scm.azurewebsites.net/DebugConsole
and run npm install:
3) Then restart the service in Azure
You can use module.exports for example:
module.exports = async function queryDatabase() {
const pg = require('pg');
//...
//...
}
if you are exporting multiple functions
for example function1, function2 then
by adding this line to the end of file we can resolve this issue.
exports.index = function1

Can't get range from a defined name

Excel 2016 (Office 365) 32 bits, 16.0.6965.2115, Visual Studio 14.0.25425.01 Update 3
I'm quite sure the statement below used to work, but now it doesn't work anymore:
var range = ctx.workbook.names.getItem("Countries").getRange();
I get an error stating that there is no support for getRange method, but it should be supported as documented here.
What am I'm doing wrong?
--- EDIT: this is the code I'm using ---
function paintRange() {
Excel.run(function (ctx) {
var range = ctx.workbook.names.getItem("Countries").getRange();
range.format.fill = "green";
return ctx.sync();
}).catch(function (error) {
app.showNotification("Error", error);
})
}
paintRange is attached to a button. There is a global scope defined name called Countries.
I don't have any more details of the error besides the one I mentioned, I also tried opening the quick watch window to get more clues.
UPDATE: The issue is fixed with an update to the CDN. You should be able to use namedItem.getRange() now. Thanks for reporting the issue, and allowing us to do a quick turn-around on it.
================
Felipe, looks like you're absolutely right. This is definitely a bug. Let me talk to the right folks to get this regression fixed as soon as we can. I'll see if we can put in some processes to avoid this in the future, as well.
From an immediate-workaround perspective, two options:
Use the BETA CDN (esp if it's for an in-development add-in, rather than a production one). That URL is: https://appsforoffice.microsoft.com/lib/beta/hosted/office.js
Do a temporarily filling in of the inadvertently-removed getRange functionality. Inside of Office.initialize, include the following code:
if (!Excel.NamedItem.prototype.getRange) {
Excel.NamedItem.prototype.getRange=function () {
return new Excel.Range(this.context,
OfficeExtension.ObjectPathFactory.createMethodObjectPath(
this.context, this, "GetRange",
OfficeExtension.OperationType.Read, [], false, true, null
)
);
};
}
The workaround in #2 should not cause harm even after the functionality is restored, but I would none-the-less recommend making a mental note to remove this after we've fixed the issue. I'll update this thread once we have fixed the underlying bug, hopefully within a weeks' time (as a very rough estimate, pending any complications that might delay it).
Thanks for bringing it to our attention -- both the individual bug, and the underlying process that let the regression to this one API go unnoticed.

Standalone PHP script using Expression Engine

Is there a way to make a script where I can do stuff like $this->EE->db (i.e. using Expression Engine's classes, for example to access the database), but that can be run in the command line?
I tried searching for it, but the docs don't seem to contain this information (please correct me if I'm wrong). I'm using EE 2.4 (the link above should point to 2.4 docs).
The following article seems to have a possible approach: Bootstrapping EE for CLI Access
Duplicate your index.php file and name it cli.php.
Move the index.php file outside your DOCUMENT_ROOT. Now, technically, this isn’t required, but there’s no reason for prying
eyes to see your hard work so why not protect it.
Inside cli.php update the $system_path on line 26 to point to your system folder.
Inside cli.php update the $routing['controller'] on line 96 to be cli.
Inside cli.php update the APPPATH on line 96 to be $system_path.'cli/'.
Duplicate the system/expressionengine directory and name it system/cli.
Duplicate the cli/controllers/ee.php file and name it cli/controllers/cli.php.
Finally, update the class name in cli/controllers/cli.php to be Cli and remove the methods.
By default EE calls the index method, so add in an index method to do what you need.
#Zenbuman This was useful as a starting point although I would add I had issues with all of my requests going to cli -> index, whereas I wanted some that went to cli->task1, cli->task2 etc
I had to update *system\codeigniter\system\core\URI.php*so that it knew how to extract the parameters I was passing via the command line, I got the code below from a more recent version of Codeigniter which supports the CLI
// Is the request coming from the command line?
if (php_sapi_name() == 'cli' or defined('STDIN'))
{
$this->_set_uri_string($this->_parse_cli_args());
return;
}
// Let's try the REQUEST_URI first, this will work in most situations
and also created the function in the same file
private function _parse_cli_args()
{
$args = array_slice($_SERVER['argv'], 1);
return $args ? '/' . implode('/', $args) : '';
}
Also had to comment out the following in my cli.php file as all routing was going to the index method in my cli controller and ignoring my parameters
/*
* ~ line 109 - 111 /cli.php
* ---------------------------------------------------------------
* Disable all routing, send everything to the frontend
* ---------------------------------------------------------------
*/
$routing['directory'] = '';
$routing['controller'] = 'cli';
//$routing['function'] = '';
Even leaving
$routing['function'] = '';
Will force requests to go to index controller
In the end I felt this was a bit hacky but I really need to use the EE API library in my case. Otherwise I would have just created a separate application with Codeigniter to handle my CLI needs, hope the above helps others.
I found #Zenbuman's answer after solving my own variation of this problem. My example allows you to keep the cron script inside a module, so if you need your module to have a cron feature it all stays neatly packaged together. Here's a detailed guide on my blog.

SWF SecurityError: Error #2000: No active security context

Hi
I have a flash image gallery that worked just fine, until few days a go it stopped loading the images. the debugger throws this error :
SecurityError: Error #2000: No active security context.
can someone explain what can be the cause?
I've run into this problem when working with loading images where the path is located in an external XML file. So... I load the XML get the path from it but then the problem I had was I was loading 30+ images and the error was popping up only 6 times so.. I had no idea which file locations where the bad ones.
If you want flash to out put more info than just :
SecurityError: Error #2000: No active security context.
Add this event listener to your Loader:
loader.contentLoaderInfo.addEventListener(IOErrorEvent.IO_ERROR, ioErrorHandler);
and finally this function:
protected function ioErrorHandler(e:IOErrorEvent):void{
trace(e.text);
}
With this in place your Security Error will convert to a URL Not Found Error with the file location you supplied. With this information in hand it should be easier for you to debug the problem.
Error #2035: URL Not Found. URL: file:////Volumes/Macintosh%20HD/Users/cleanshooter/Documents/Website%20/here/there/everywhere/30805/filename.jpg
I faced this issue before,the final conclusion was related to incorrect image path or name
Did your images extensions change, possibly from like .jpg to .JPG or something?
Typically this is called if there is a problem with your external media. Here's a workaround for it, but I typically try and solve versus make it go away.
setTimeout( function():void{fileReference.load();}, 1);
Hope this helps.
I ran across this issue and used the above setTimeout example but for a slightly different purpose. I was calling a php script that hit Twitter and got the same security issue in Flash debug player. I just wanted to add my example which builds on the above to show how you can use this "workaround" for URLLoader as well as fileReference.
var myXMLLoader:URLLoader = new URLLoader();
var urlStr:String = "http://www.yourdomain.com/php/twitter.php";
var myVariables:URLVariables = new URLVariables();
myVariables.twitterID = "yourtwitterID";
var myURLRequest:URLRequest = new URLRequest(urlStr)
myURLRequest.data = myVariables;
setTimeout(function():void { myXMLLoader.load( myURLRequest ); }, 1);
myXMLLoader.addEventListener(Event.COMPLETE, onXMLLoadHandler);
You need to handle the error:
loader.contentLoaderInfo.addEventListener(HTTPStatusEvent.HTTP_STATUS, onHTTPError);
protected function onHTTPError(e:HTTPStatusEvent):void{
trace("HTTPError"+e.status);
}
This way it will handle the error and works fine.
In response to headwinds:
In AS3 you need to import flash.utils.setTimeout. The syntax for setTimeout is setTimeout(A, B, ...rest);
Where B is the function to get called afterwards,
A is the delay in ms (e.g. 1000 for a second)
and C is any number of parameters you need to provide for the function, separated by a comma.
E.g.
import flash.utils.setTimeout;
// package, etc
//main function
setTimeout(respond, 500, true, false);
private function respond(A : Boolean, B : Boolean) : void {
var result : Boolean = A == B;
trace(result);
}

Resources