Chrome run fails in Azure Functions: An attempt was made to access a socket in a way forbidden by its access permissions - azure

I wrote a web bot that uses Selenium framework to crawl. Installed ChromeDriver 72.0.3626.69 and also downloaded Chromium 72.0.3626.121. The app initializes ChromeDriver with this included Chromium binary (and NOT a locally installed Chrome binary). All this perfectly works on my machine locally.
I've been attempting now to port the app to Azure Functions. I wrote a function, tested it, and it works fine locally. But once I publish it to Azure Functions it fails due to about 182 errors of type:
An attempt was made to access a socket in a way forbidden by its
access permissions
I know this happens due to exceeding the TCP connection limits of Azure sandbox, but the only attempt here was to create an instance of ChromeDriver (not even navigate anywhere yet!)
Here is a screenshot of Azure Function call log.
That error appears about 182 times in a row, and that's basically just an attempt to create a browser instance (or ChromeDriver instance, to be precise - can't be sure if that's Chromium or ChromeDriver causing the issue).
The question: Have anyone experienced issues with ChromeDriver/Chromium creating so many (obviously excessive) connections when launching? And what might help to avoid this.
If that's of any help, this is basically a piece of code that crashes on the last line:
ChromeOptions options = new ChromeOptions();
options.BinaryLocation = this.chromePath;
options.AddArgument("no-sandbox");
options.AddArgument("disable-infobars");
options.AddArgument("--disable-extensions");
if (this.headlessMode)
{
options.AddArgument("headless");
}
options.AddUserProfilePreference("profile.default_content_setting_values.images", 2);
Log.LogInformation("Chrome options compiled. Creating ChromeDriverService...");
var driverService = ChromeDriverService.CreateDefaultService(this.driverPath);
driver = new ChromeDriver(driverService, options, timeout);

I believe you are running this function in a Windows Function App which is subject to quite a few limitations as described in this wiki.
But when running on Linux, functions are basically run in a docker container, removing most of these restrictions that windows has. I believe what you are trying should be possible there.
You could either just deploy your function to a Linux Function App or even build a container and use that directly as well.

Related

Selenium with Python. "Bluetooth: bluetooth_adapter_winrt.cc:1076 " . Error [duplicate]

I have updated Selenium but the error keeps occurring even though the web page loads. However, in some instances, the driver starts but it is stagnant. Is this causing an issue and if so, how do I resolve it?
[11556:9032:0502/152954.314:ERROR:device_event_log_impl.cc(162)] [15:29:54.314] Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter failed.
This error message...
ERROR:device_event_log_impl.cc(162)] [15:29:54.314] Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter failed.
...implies that ScopedClosureRunner on_init failed in BluetoothAdapterWinrt::OnGetDefaultAdapter().
Analysis
This error is defined in bluetooth_adapter_winrt.cc as follows:
void BluetoothAdapterWinrt::OnGetDefaultAdapter(
base::ScopedClosureRunner on_init,
ComPtr<IBluetoothAdapter> adapter) {
DCHECK_CALLED_ON_VALID_THREAD(thread_checker_);
if (!adapter) {
BLUETOOTH_LOG(ERROR) << "Getting Default Adapter failed.";
return;
}
Solution
Ensure that:
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v84.0 level.
Chrome is updated to current Chrome Version 84.0 level. (as per ChromeDriver v84.0 release notes)
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.
Additional considerations
However it was observed that this error can be supressed by running Chrome as root user (administrator) on Linux. but that would be a deviation from the documentation in ChromeDriver - WebDriver for Chrome where it is mentioned:
A common cause for Chrome to crash during startup is running Chrome as root user (administrator) on Linux. While it is possible to work around this issue by passing '--no-sandbox' flag when creating your WebDriver session, i.e. the ChromeDriver session as such a configuration is unsupported and highly discouraged.
Ideally, you need to configure your environment to run Chrome as a regular user instead.
Suppressing the error
Finally, as per the documentation in Selenium Chrome Driver: Resolve Error Messages Regarding Registry Keys and Experimental Options these error logs can be supressed by adding the argument:
excludeSwitches: ['enable-logging']
So your effective code block will be:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")
I had simmilar problems
ConnectionResetError: [WinError 10054] An existing connection was
forcibly closed by the remote host and
Bluetooth: bluetooth_adapter_winrt.cc:1055 Getting Default Adapter
failed.
Both of them disapeared after running cmd as an administrator. I don't know what is the exact cause of this issue but for me it seems that's a lack of privs while running selenium.
If anyone could explain why it is happening would be great.
Simply switching on my device's Bluetooth solved the problem... Don't know the reason behind it
I was getting the same error. On a code that was working yesterday.
The Code is available at this url at this moment https://youtu.be/0kLoVGLTISg?list=PLUDwpEzHYYLvx6SuogA7Zhb_hZl3sln66&t=4073
https://github.com/Microsoft/vscode-python/issues/3252 Found the Resolution hint over here in the comments section, along with https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp,
suggesting that, we cannot run on "Pycharm"/VSCode using right click -> run from within the class level, we need to run it from the module level i.e. outside the class level, since setUpClass() method is not executed when running from inside of the class.

Stackdriver-trace on Google Cloud Run failing, while working fine on localhost

I have a node server running on Google Cloud Run. Now I want to enable stackdriver tracing. When I run the service locally, I am able to get the traces in the GCP. However, when I run the service as Google Cloud Run, I am getting an an error:
"#google-cloud/trace-agent ERROR TraceWriter#publish: Received error with status code 403 while publishing traces to cloudtrace.googleapis.com: Error: The request is missing a valid API key."
I made sure that the service account has tracing agent role.
First line in my app.js
require('#google-cloud/trace-agent').start();
running locally I am using .env file containing
GOOGLE_APPLICATION_CREDENTIALS=<path to credentials.json>
According to https://github.com/googleapis/cloud-trace-nodejs These values are auto-detected if the application is running on Google Cloud Platform so, I don't have this credentials on the gcp image
There are two challenges to using this library with Cloud Run:
Despite the note about auto-detection, Cloud Run is an exception. It is not yet autodetected. This can be addressed for now with some explicit configuration.
Because Cloud Run services only have resources until they respond to a request, queued up trace data may not be sent before CPU resources are withdrawn. This can be addressed for now by configuring the trace agent to flush ASAP
const tracer = require('#google-cloud/trace-agent').start({
serviceContext: {
service: process.env.K_SERVICE || "unknown-service",
version: process.env.K_REVISION || "unknown-revision"
},
flushDelaySeconds: 1,
});
On a quick review I couldn't see how to trigger the trace flush, but the shorter timeout should help avoid some delays in seeing the trace data appear in Stackdriver.
EDIT: While nice in theory, in practice there's still significant race conditions with CPU withdrawal. Filed https://github.com/googleapis/cloud-trace-nodejs/issues/1161 to see if we can find a more consistent solution.

SqlDataProvider connection string in Suave on Azure

I can't get SqlDataProvider to work when executed in a fsx script which is running in an Azure Web Site.
I have started from the samples that Tomas Petrecek has here: https://github.com/tpetricek/Dojo-Suave-FsHome.
In short it is a FSX script that is executed using the IIS httpPlatformHandler so that all http requests to my Azure Web site is forwarded to my F# script.
The F# Script use Suave to handle the requests.
When I tried adding some database access to my HTTP handlers I got into problems.
The problematic code looks like this:
[<Literal>]
let connStr = "Server=(localdb)\\v11.0;Initial Catalog=My_Database;Integrated Security=true;"
[<Literal>]
let resolutionFolder = __SOURCE_DIRECTORY__
FSharp.Data.Sql.Common.QueryEvents.SqlQueryEvent |> Event.add (printfn "Executing SQL: %s")
// the following line fails when executing in azure
type db = SqlDataProvider<connStr, Common.DatabaseProviderTypes.MSSQLSERVER, ResolutionPath = resolutionFolder>
let saveData someDataToSave =
let ctx = db.GetDataContext(Environment.GetEnvironmentVariable("SQLAZURECONNSTR_QUERIES"))
.....
/// code using the context here
This works just fine when I run it locally, but when I deploy it to the azure site it will fail at the line where the type dbis created.
The error message is (line 70 is the line that has the type db = ...:
D:\home\site\wwwroot\app.fsx(70,11): error FS3033: The type provider
'FSharp.Data.Sql.SqlTypeProvider' reported an error: A network-related
or instance-specific error occurred while establishing a connection to
SQL Server. The server was not found or was not accessible. Verify
that the instance name is correct and that SQL Server is configured to
allow remote connections. (provider: SQL Network Interfaces, error: 52
- Unable to locate a Local Database Runtime installation. Verify that SQL Server Express is properly installed and that the Local Database
Runtime feature is enabled.)
The design-time database in the connStr is not available in the azure site, but I thought this is why we have the GetDataContext overload that takes a connection string to be used at run-time?
Is it because it is running as a script and not as compiled code that it is trying to access the database when creating the TypeProvider?
If yes, does it mean that my only option is to compile and provide the database code as a compiled assembly that I load and use in my Suave FSX script?
Reading the connection string from a config file does not work very well as this is in a azure site. I really need to get the connection string from an environment variable (which is set in the azure management interface).
Hmm, this is a bit unfortunate - as #Fyodor mentioned in the comments, the problem is that the script-based deployment to Azure actually compiles the script on the Azure machine - and so you need to have a statically-resolved connection string that works on Azure.
There are two options:
Use compiled project instead. If you compile your F# code locally and deploy the compiled code to Azure it will work. Sadly, there are no good samples for that.
Do some clever trick to make the connection string accessible to the script at compile time.
Send a PR to the SQL provider so that you can give it the name of an environment variable and it reads the connection string from there.
I think (3) would actually be quite nice and useful feature.
I'm not necessarily sure what the best way to do (2) would be. But I think you might be able to modify app.azure.fsx so that it creates a file (say connection.fsx) that contains something like:
module Connection
let [<Literal>] ConnString = "<Contents of SQLAZURECONNSTR_QUERIES>"
Then app.fsx could load this script and use Connection.ConnString in the argument of SQL type provider.

ServiceStack Facebook Authentication NullReference Exception on Vagrant Box (Ubuntu/MySql/Mono/nginx)

Long shot I guess, with the lack of real information that I am offering at this stage. I'll gladly offer up some more details on how to reproduce the issue - but wanted some fast feedback to see if there was a gotcha somewhere I was missing.
I've a simple ServiceStack hello world application, in which I'm playing with the Facebook Auth Provider:
Vanilla ServiceStack
Vanilla Facebook Auth Proivider
Vanilla User Session
Vanilla OrmLite User Repository
Vanilla OrmLite MySql Db Factory
When debugging on my local machine - on Windows 7 (and 8); everything works a treat. The service launches, the database tables are created and I can login via Facebook and records are inserted to the relevant tables.
When running the service on Ubuntu inside a Vagrant Box (running in Virtual Box as the provider for virtualization, hosted on nginx with mono-fastcgi) - the service launches correctly and I can see that the tables are created in the MySql database. When I hit /auth/facebook I am correctly forwarded to Facebook - but I hit an error when the callback to the service occurs.
This is the current output:
[Auth: 07/30/2013 13:02:47]: [REQUEST: {provider:facebook}] System.NullReferenceException: Object reference not set to an instance of an object at
ServiceStack.ServiceInterface.Auth.FacebookAuthProvider.Authenticate (ServiceStack.ServiceInterface.IServiceBase,ServiceStack.ServiceInterface.Auth.IAuthSession,ServiceStack.ServiceInterface.Auth.Auth) <0x0061e> at
ServiceStack.ServiceInterface.Auth.AuthService.Authenticate (ServiceStack.ServiceInterface.Auth.Auth,string,ServiceStack.ServiceInterface.Auth.IAuthSession,ServiceStack.ServiceInterface.Auth.IAuthProvider) <0x000a7> at
ServiceStack.ServiceInterface.Auth.AuthService.Post (ServiceStack.ServiceInterface.Auth.Auth) <0x00303> at
ServiceStack.ServiceInterface.Auth.AuthService.Get (ServiceStack.ServiceInterface.Auth.Auth) <0x00013> at (wrapper dynamic-method) object.lambda_method (System.Runtime.CompilerServices.Closure,object,object) <0x0004f> at
ServiceStack.ServiceHost.ServiceRunner`1<ServiceStack.ServiceInterface.Auth.Auth>.Execute (ServiceStack.ServiceHost.IRequestContext,object,ServiceStack.ServiceInterface.Auth.Auth) <0x00416>
It is clearly reaching the Service (which I'm accessing via localhost:8080 which maps through to the guest machine on port 80); as the error is wrapped nicely in ServiceStack output.
I don't suppose anyone has any clues?
Okay after an evening of investigation - I've found the root cause.
Line 51 of FacebookAuthProvider.cs calls off to Line 28 of WebRequestExtensions.cs - which in turn calls Line 227 of WebRequestExtensions.cs.
This method call fails at line 255-ish - essentially because Mono by default doesn't trust any SSL certificates by default: as explained here..
Instead of figuring out the correct configuration for Mono - I've taken the nasty route (for the time being at least); of using the following line in my AppHostBase.Configure implementation:
F#
System.Net.ServicePointManager.ServerCertificateValidationCallback <- new RemoteCertificateValidationCallback(fun _ _ _ _ -> true)
C#
System.Net.ServicePointManager.ServerCertificateValidationCallback += (a, b, c, d) => { return true; };
I am now up and running (like a fully-operational Death Star).

Intermittent IE timeouts and COMExceptions when running Watin tests with Nant and CruiseControl

We’ve been using Watin and CruiseControl.net for a few weeks now and most of the time they work together well. We have not had any problems running the tests on our developer machines. We are also able to run the tests interactively without problems when logged into the CI server.
Most of the time there are also no problems when the tests are executed under CruiseControl but this is not always the case as we’ve recently been seeing intermittent errors. The errors seem to come and go somewhat randomly but when an error does occur, it’s always one of the following:
WatiN.Core.Exceptions.TimeoutException: Timeout while Internet Explorer busy
System.Runtime.InteropServices.COMException: Creating an instance of the COM component with CLSID {0002DF01-0000-0000-C000-000000000046} from the IClassFactory failed due to the following error: 800704a6
Our CI server environment is:
Windows Server 2008 R2
IIS 7.5
IE 8
Watin 2.0
CruiseControl 1.6.7981.1. This is running as a service which logs in as a user on our domain because the tests need to access resources on the domain.
The 'randomly' failing tests create their IE instances as follows:
[TestMethod]
public void SomeTest()
{
using (var browser = new IE())
{
// run tests here
}
}
I also tried creating the IE in a new process as follows:
[TestMethod]
public void SomeTest()
{
using (var browser = new IE( true))
{
// run tests here
}
}
But when I did that, all of our tests failed with a “WatiN.Core.Exceptions.BrowserNotFoundException: Could not find an IE window matching constraint: Timeout while waiting to attach to newly created instance of IE.. Search expired after '30' seconds”
So, I have two questions:
Can anyone tell me how I can stop the timeouts and com exceptions?
Can anyone explain why IE(true) didn’t work at all?
TIA,
Mike
See this answer by Carl Hörberg:
Running Watin on TeamCity
CruiseControl.Net and TeamCity have the same problem when running as a service and the work-around should work for both environments.
By default, the ccservice runs as Local System, does not have access to interact with the desktop UI, and the Local System account privileges are limited and that's most probably what is causing the TimeoutException, COMException, and BrowserNotFoundException from being thrown.

Resources