Azure ML release bug AZUREML_COMPUTE_USE COMMON_RUNTIME - azure

On 2021-10-13 in our application in Azure ML platform we get this new error that causes failures in pipeline steps - python module import failures - warning stack <- warning that leads to pipeline runtime error
we needed to set it to false. Why is it failing? What exactly are exact (and long term) consequences when opting out? Also, Azure ML users - do you think it was rolled out appropriately?

Try to add into your envirnoment new variable like this:
environment.environment_variables = {"AZUREML_COMPUTE_USE_COMMON_RUNTIME":"false"}

Long term (throughout 2022), AzureML will be fully migrating to the new Common Runtime on AmlCompute. Short term, this change is a large undertaking, and we're on the lookout for tricky functionality of the old Compute Runtime we're not yet handling correctly.
One small note on disabling Common Runtime, it can be more efficient (avoids an Environment rebuild) to add the environment variable directly to the RunConfig:
run_config.environment_variables["AZUREML_COMPUTE_USE_COMMON_RUNTIME"] = "false"
We'd like to get more details about the import failures, so we can fix the regression. Are you setting the PYTHONPATH environment variable to make your custom scripts importable? If so, this is something we're aware isn't working as expected and are looking to fix it within the next two weeks.

We identified the issue and have rolled out a hotfix on our end addressing the issue. There are two problems that could've caused the import issue. One is that we are overwriting the PYTHONPATH environment variable. The second is that we are not adding the python script's containing directory to python's module search path if the containing directory is not the current working directory.
It would be great if you can please try again without setting the AZUREML_COMPUTE_USE_COMMON_RUNTIME environment variable and see if the problem is still there. If it is, please reply to either Lucas's thread or mine with a minimal repro or description of where the module you are trying to import is located at in relation to the script being run and the root of the snapshot (which is the current working directory).

Related

ENSDFSTATE.dat file not found error while running exampleB1.exe using Geant4

I am new in Geant4. i installed and build GEANT4 using cmake and visual stdio 17 on windlows 10 and now i am trying to practice its given basic exapmles, So, I build first example using cmake by following the toturial given here. The example build successfully and created the exampleB1.exe file. I have set up all environment variables to data installed as given in screen shot
but facing the problem of still relating to environment vairiable is the screenshots of error
I looked this ENSDFSTATE.dat file but could not be able to solve the problem. Any suggestions/ support is highly appreciated.
It looks like your "G4ENSDFSTATEDATA" environmental variable is pointing to the wrong data set. Edit this variable to point to G4ENSDFSTATE2.2 location (i.e. D:\GEANT4\share\Geant4-10.5.1\data\G4ENSDFSTATE2.2 instead of D:\GEANT4\share\Geant4-10.5.1\data\G4PARTICLEXS1.1) and should work fine for that one. Double check your other environmental variables are correct also. Looks like G4ABLADATA should be G4ABLA3.1 for example.
Here is the list of all variables from the getting started manual (http://geant4-userdoc.web.cern.ch/geant4-userdoc/UsersGuides/InstallationGuide/html/postinstall.html):
Good luck!

Is there a way to use a global variable for the whole npm project?

I need to use a variable between node modules folder and src folder.Is there a way to use a variable to be used within the whole project?
Thanks.
Typically this is done with process environment variables. Any project can look for a common environment variable and handle that case accordingly, as well as the node modules of said project. As long as they agree on the name, they all share the same environment. Have a good default, don't force people to set this environment variable.
The environment can be set using an environment file (do not check this into source control!), on your container or cloud configuration, or even right on the command line itself.
TLOG=info npm test
That is an example that I use frequently. My project runs logging for the test cases only at alert level - there are a lot of tests so it makes the output less verbose. However, sometimes while developing I want to see all the logs, so my project is looking for an environment variable TLOG (short for "test logging") and I can set it just for that run! Also no code change is needed, which is nicer than JavaScript variables that need to be set back to original values, forget to be turned off, etc.

Running Brightway with project dir on user-defined directory

The default directory in which Brightway stores projects and all associated components is determined by appdirs. Indeed, in bw2data.projects, the project directory is set as:
data_dir = appdirs.user_data_dir(LABEL, "pylca")
For example, for my Windows install , this is C:\users\me\AppData\Local\pylca\Brightway3.
I would like for one of my projects to be on an external network-based disk. This is for a used project, not just for cold storage. Is there functionality within Brightway to change the location of a project?
Yes, and the best way to do this is in the activation script for your project-specific virtual environment. See the FAQs (and please report an issue if more detail is needed or something is wrong):
https://docs.brightwaylca.org/faq.html#how-do-i-find-where-my-data-is-saved
https://docs.brightwaylca.org/faq.html#setting-brightway2-dir-in-a-virtual-environment
As an alternative procedure if you want to change BRIGHTWAY2_DIR in Python, this works:
import os
os.environ['BRIGHTWAY2_DIR']='path/to/my/other/dir'
from brightway2 import *
Despite interesting leads such as this one on reload, I have been unable to make this work if there was a brightway2 import before setting the BRIGHTWAY2_DIR environment variable.

Perl: libapt-pkg-perl AptPkg::Cache->new strange behaviour under precise

I have a very strange problem with the constructor of AptPkg::Cache object in the precise package of libapt-pkg-perl (v. 0.1.25).
The perl script is designed to download a debian package for three different architectures (i386, armel, armhf). For each architecture I do the following:
Configure AptPkg::Config '$_config' with the right parameters and package-lists for the desired architecture.
Create the cache object with AptPkg::Cache->new .
Call the method AptPkg::Cache->policy to create the AptPkg::Policy object.
Call the method AptPkg::Policy->candidate("program-name") .
Download the package for the selected architecture.
This works very well with Ubuntu Lucid, but with Ubuntu Precise I can only download the package for the first architecture defined. For the other two architectures there will be no installation candidate (method AptPkg::Policy->candidate("Package-Name") doesn't return an object).
I tried to build a workaround and I found one solution how the script works for all three architectures, without problems, in precise:
If I create the cache object (with AptPkg::Cache->new) twice in a row it works and the script downloads the debian package for all three architectures:
my $cache = AptPkg::Cache->new;
$cache = AptPkg::Cache->new;
I'm sure that the problem has something to do with the method AptPkg::Cache->new because I checked everything else, what could cause the problem, twice. All config-variables are set correctly and I even get a different Hash for AptPkg::Cache->new for each architecture, but it seems that I am overlooking something important.
I'm not very familiar with perl, so I am asking you guys if someone can explain why the script works with the workaround but not without it. Further it looks quite strange if you have the same line of code twice in your script.
Maybe you hit this bug - https://bugs.launchpad.net/ubuntu/+source/libapt-pkg-perl/+bug/994509
There is a script there to test if you're affected. If it's something else consider submitting a bug report.
edit: Just saw this is 11 months old :/

Runtime error R6034 in embedded Python application

I am working on an application which uses Boost.Python to embed the Python interpreter. This is used to run user-generated "scripts" which interact with the main program.
Unfortunately, one user is reporting runtime error R6034 when he tries to run a script. The main program starts up fine, but I think the problem may be occurring when python27.dll is loaded.
I am using Visual Studio 2005, Python 2.7, and Boost.Python 1.46.1. The problem occurs only on one user's machine. I've dealt with manifest issues before, and managed to resolve them, but in this case I'm at a bit of a loss.
Has anyone else run into a similar problem? Were you able to solve it? How?
The problem was caused by third-party software that had added itself to the path and installed msvcr90.dll in its program folder. In this case, the problem was caused by Intel's iCLS Client.
Here's how to find the problem in similar situations:
Download Process Explorer here.
Start your application and reproduce runtime error R6034.
Start Process Explorer. In the "View" menu go to "Lower Pane View" and choose "DLLs".
In the top pane, locate your application and click on it. The bottom pane should show a list of DLLS loaded for your application.
Locate "msvcr??.dll" in the list. There should be several. Look for the one that is not in the "winsxs" folder, and make a note of it.
Now, check the path just before your application runs. If it includes the folder you noted in step 5, you've probably found the culprit.
How to fix the problem? You'll have to remove the offending entry from the path before running your program. In my case, I don't need anything else in the path, so I wrote a simple batch file that looks like this:
path=
myprogram.exe
That's it. The batch file simply clears the path before my program runs, so that the conflicting runtime DLL is not found.
This post elaborates on #Micheal Cooper and #frmdstryr and gives a better alternative than my earlier answer.
You can put the following in front of a python script to purge the problematic entries.
import os, re
path = os.environ['PATH'].split(';')
def is_problem(folder):
try:
for item in os.listdir(folder):
if re.match(r'msvcr\d\d\.dll', item):
return True
except:
pass
return False
path = [folder for folder in path if not is_problem(folder)]
os.environ['PATH'] = ';'.join(path)
For the vim with YouCompleteMe case, you can put the following at the top of your vimrc:
python << EOF
import os, re
path = os.environ['PATH'].split(';')
def is_problem(folder):
try:
for item in os.listdir(folder):
if re.match(r'msvcr\d\d\.dll', item):
return True
except:
pass
return False
path = [folder for folder in path if not is_problem(folder)]
os.environ['PATH'] = ';'.join(path)
EOF
A more general solution is:
import os
os.environ['path'] = ";".join(
[path for path in os.environ['path'].split(";")
if "msvcr90.dll" not in map((lambda x:x.lower()), os.listdir(path))])
(I had the same problem with VanDyke SecureCRT)
(This might be better as a comment than a full answer, but my dusty SO acct. doesn't yet have enough rep for that.)
Like the OP I was also using an embedded Python 2.7 and some other native assemblies.
Complicating this nicely was the fact that my application was a med-large .Net solution running on top of 64-Bit IIS Express (VS2013).
I tried Dependency Walker (great program, but too out of date to help with this), and Process Monitor (ProcMon -- which probably did find some hints, but even though I was using filters the problems were buried in thousands of unrelated operations, better filters may have helped).
However, MANY THANKS to Michael Cooper! Your steps and Process Explorer (procexp) got me quickly to a solution that had been dodging me all day.
I can add a couple of notes to Michael's excellent post.
I ignored (i.e. left unchanged) not just the \WinSxS\... folder but also the \System32\... folder.
Ultimately I found msvcr90.dll being pulled in from:
C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64
Going through my Path I found the above and another, similar directory which seemed to contain 32-bit versions. I removed both of these, restarted and... STILL had the problem.
So, I followed Michael's steps once more, and, discovered another msvcr90.dll was now being loaded from:
C:\Program Files\Intel\iCLS Client\
Going through my Path again, I found the above and an (x86) version of this directory as well. So, I removed both of those, applied the changes, restarted VS2013 and...
No more R6034 Error!
I can't help but feel frustrated with Intel for doing this. I had actually found elsewhere online a tip about removing iCLS Client from the Path. I tried that, but the symptom was the same, so, I thought that wasn't the problem. Sadly iCLS Client and OpenCL SDK were tag-teaming my iisexpress. If I was lucky enough to remove either one, the R6034 error remained. I had to excise both of them in order to cure the problem.
Thanks again to Michael Cooper and everyone else for your help!
Using Michael's answer above, I was able to resolve this without a bat file by adding:
import os
# Remove CLS Client from system path
if os.environ['PATH'].find("iCLS Client")>=0:
os.environ['PATH'] = "".join([it for it in os.environ['PATH'].split(";") if not it.find("iCLS Client")>0])
to the main python file of the application. It just makes sure system path didn't include the paths that were causing the issue before the libraries that loaded the dll's were imported.
Thanks!
This post elaborates on #Micheal Cooper and #frmdstryr.
Once you identified the problematic PATH entries, you can put the following in front of a
python script, assuming here that iCLS Client and CMake are problematic.
import os
for forbidden_substring in ['iCLS Client', 'CMake']:
os.environ['PATH'] = ';'.join([item for item in os.environ['PATH'].split(';')
if not item.lower().find(forbidden_substring.lower()) >= 0])
Concerning the vim with YouCompleteMe case, you can put the following at the top of your vimrc:
python << EOF
import os
for forbidden_substring in ['iCLS Client', 'CMake']:
os.environ['PATH'] = ';'.join([item for item in os.environ['PATH'].split(';')
if not item.lower().find(forbidden_substring.lower()) >= 0])
EOF
If none of these solutions is applicable for you, you can try to remove the problem causing
entries from you PATH manually, but you want to make sure you don't break anything else on your
system that depends on these PATH entries. So, for instance, for CMake you could try to remove
its PATH entry, and only put a symlink (or the like) pointing to the cmake.exe binary into some
other directory that is in your PATH, to make sure cmake is still runnable from anywhere.
Thanks for the solution. I just little modified this sample code as the path variable in my system contains the string "ICLS CLIENT" instead of "iCLS Client"
import os
# print os.environ['PATH']
# Remove CLS Client from system path
if os.environ['PATH'].find("iCLS Client") >= 0 or os.environ['PATH'].find("ICLS CLIENT") >= 0:
os.environ['PATH'] = "".join([it for it in os.environ['PATH'].split(";") if not (it.find("iCLS Client")>0 or it.find("ICLS CLIENT")>0)])
I also had the same problem with embedding Python27.dll from a C-program using the Universal-CRT.
A <PYTHON_ROOT>\msvcr90.dll was the offender. And <PYTHON_ROOT> is off-course in my PATH. AFAICS the only users of msvcr90.dll are the PyWin32 modules
<PYTHON_ROOT>\lib\site-packages\win32\win32*.pyd.
The fix was just move <PYTHON_ROOT>\msvcr90.dll to that directory.
PS. PyWin32 still has this as an issue 7 years later!
In my case the rebuilding of linked libraries and the main project with similar "Runtime execution libraries" project setting helped. Hope that will be usefull for anybody.
In my case, I realised the problem was coming when, after compiling the app into an exe file, I would rename that file. So leaving the original name of the exe file doesn't show the error.
The discussion on this page involves doing things way far advanced above me. (I don't code.) Nevertheless, I ran Process Explorer as the recommended diagnostic. I found that another program uses and needs msvcr90.dll in it's program folder. Not understanding anything else being discussed here, as a wild guess I temporarily moved the dll to a neighboring program folder.
Problem solved. End of Runtime error message.
(I moved the dll back when I was finished with the program generating the error message.)
Thank you all for your help and ideas.
Check any library having user specified path by Process Explorer. It is not necessary must be msvcr??.dll
I solved same problem except I run Python 3. Present solutions not helped because they not indicate unusual paths of msvcr90.dll. I debug code step by step inside till error dialog appears after rows (called when my code was importing PyTables module):
import ctypes
ctypes.cdll.LoadLibrary('libbz2.dll')
Then Process Explorer helps to find path to old libbz2.dll caused the problem (steps 3, 4 of #Micheal Cooper algorithm)
Adding this answer for who is still looking for a solution. ESRI released a patch for this error. Just download the patch from their website (no login required), install it and it will solve the problem. I downloaded the patch for 10.4.1 but there are maybe patches for other versions also.

Resources