I only seem able to integrate my ADFv2 instance with public repos in my organisation, which is undesirable.
In the documentation (https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/data-factory/source-control.md#author-with-github-integration), it states that:
..can use both public and private GitHub repositories with Data Factory as long you have read and write permission to the repository..
So, i complete the following steps:
Navigate to my GitHub Org.
Create a new "Private" repo, initialised w/ README.MD.
Create a new "Internal" repo, initialised w/ README.MD. (this is GitHub Enterprise Cloud)
In the Azure Portal, create a new ADFv2 instance without Git integration.
Click "Set up code repository" in the ADFv2 instance.
Choose, RepoType: GitHub - Authenticate w. GitHub pop-up.
Enter org name in to the GitHub Account section.
Git repo name: No results found
🤷🏼‍♂️
The solution turned out to be the need to grant access as an Authorized OAuth App to the organisation. I did not have permission to do this.
You can then find it under 'Applications' once done.. but i think when you first connect up your ADF to GitHub, it prompts you if you would like to grant it.
I was not seeing this due to permissions on the Org.
Related
I'm running into an error code in the Pull request manager Hub in the Azure DevOps:
There was an error during the extension load: TFS.WebApi.Exception: TF401027: You need the Git 'GenericRead' permission to perform this action. Details: identity '...', scope 'repository'.
which leaves me a bit baffling since the team lead and manager is already sure that I have permission to view pull request, I could even comment and fix pull request that I was added on, but the manager hub is where it refused to show up for me.
the permission management
enter image description here
This error occurs when a user with a stakeholder license tries to create a repository
Solution: Upload the license of the user from “StakeHolder” to “Basic” or get him MSDN Visual Studio Professional or Enterprise subscription
By default stakeholders won’t have access to repos but they can try to create repository from organization settings..
I have a problem creating a new repository in Cloud Source Repository (console or gcloud).
In the gcloud case, when I run the following command using either an user account with roles/owner or a service account with roles/source.admin:
gcloud source repos create new-repo
I get the following error:
ERROR: (gcloud.source.repos.create) ResponseError:
status=[PERMISSION_DENIED], code=[403], message=[User [xxx#xxx] does
not have permission to access projects instance [PROJECT_ID] (or it
may not exist): The caller does not have permission].
The project currently has no repositories associated to it. I was able to create one earlier (through the console) which I then deleted before using. I have been unable to create any subsequent repositories since.
Would love to hear some suggestions if anyone else has faced a similar issue.
As you mentioned to have the proper roles on the account this issue is due to not having a billing account linked to the project.
The API methods used to create a repository require the billing on the project to be enabled so to solve it if this is the issue you can perform the following steps
Go to https://console.cloud.google.com/billing
Select the project where you want the repository to reside
Click on Link a Billing Account
A Popup message will show, over there click on Create Billing Account
Follow the wizard
So the problem is that I created Azure Data Factory with git enabled on the portal. After the resource was validated and created, and after clicking on Author and Monitor, the "Copy Data" feature seems to be disabled as shown in the picture below:
When hovered on the "Copy Data": This feature is disabled in 'Data Factory' mode as your factory has GIT enabled. Please switch to 'GIT' mode to use this.
After clicking on the Manage, the disconnect button seems to be disabled again as shown in the picture below.
How do I enable the button and disconnect my GitHub to enable "Copy data" feature?
Update:
Nandan said to check on Author but it is disabled as well.
On the home page of Azure Data Factory, can you see the option "Git repo settings" at the upper right corner? If yes, you can try to remove the GIT connection via this option.
Below are the tickets for the similar questions, you can reference:
Enable Publishing in 'Data Factory' mode
You are not allowed to make changes or publish from 'Data Factory' mode as your factory has GIT enabled
You can disable the GIT by disconnecting via Manage option as seen below:
But initially you would need to go to Author section to hover to drop down to change from Data factory mode to Git Mode:
The above images are via account which has access on GIT as well as ADF.
I logged in via an Account which has contributor access on ADF but no access on the repo and when I logged into ADF I got the below warning which got blocked in pop up of explorer :
And when I clicked on continue, and when I went to Manage section ; the disconnect option was disabled for me as seen below
So I added my account to Repo and after sometime I was able to manage the GIT in ADF
I'm having the exact same issues as the OP. I have an username#mybusiness.itservices user on Azure and a myname#gmail private GitHub account, and it's impossible to invite the Azure user because they don't have an email (they are just for administering Azure resources).
Integration with GitHub: 1) didn't work at all when configured at the launch time of ADF, 2) worked for one session when created within a new ADF instance, and 3) doesn't work when logging back into ADF.
I get the exact same issues: cannot disconnect from git:
and can't change into Git mode to do any work in ADF:
Naturally, the GitHub account has ADF authorised but there's no collaborator on the repo because I can't invite a user who doesn't have email:
None of the linked resources deal with this issue because there's no way to disconnect Git from ADF unless the "disconnect" button is no longer greyed-out. I would really appreciate any more ideas.
I ran into this issue myself and was able to resolve it. It occurred because the collaboration branch was deleted. In the browser, I was unable to select Git mode because it was confused.
To fix it, I turned to PowerShell. You need Powershell with the Az cmdlets installed.
https://learn.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-5.3.0
You'll need to know:
Subscription name in Azure
Data Factory name
Collaboration branch in github
Root folder in github
Account name for github
Repository name in github
Fire up Powershell with the Azure cmdlets and do like so...
Connect-AzAccount
Set-AzContext "<yoursubscriptionname>"
$dataFactory = Get-AzDataFactoryV2 -Name "<your data factory name>"
Set-AzDataFactoryV2 -InputObject $dataFactory -CollaborationBranch "<branch>" -RootFolder "<github root folder>" -AccountName "<github account>" -RepositoryName "<github repository>" -HostName "https://github.com"
When I ran that series of Powershell commands, it asked me if I wanted to overwrite, I said yes. Then I was able to deal with Git in Data Factory again.
I am setting up Azure Pipelines, I have few that get sources from GitHub and trying to setup pipelines to reach TFS on intranet for some time.
I started anew, I created a new PAT token in TFS with full access.
I created a new Service Connection of type: “Azure Repos/Team Foundation Server” using this URL: https://tfs.myCie.com/defaultcollection/MyProject and the PAT token from TFS above.
I created a new pipeline, selecting Other Git and using the new service connection. I didn’t know what to use as default branch as I don’t see any branches on TFS so I used master.
I used the empty job option and entered again the service connection name and the master branch. When I tried to select the YAML template, it wasn’t able to save it to a repo, should I have a different repo for Yaml files to TFS?
I paid attention that my .proxy file was filled and the .proxybypass file had the TFS server URL.
When I run the pipeline, early on the UI gives a timeout but after 6-7 minutes, I have logs that says:
Syncing repository: repository (ExternalGit)
##[debug]repository url=https://tfs.myCie.com/defaultcollection/myProject/
##[debug]targetPath=D:\Agent_work\19\s
git version 2.26.2.windows.1
git remote add origin https://tfs.myCie.com/defaultcollection/myProject/
##[debug]Finished process 6888 with exit code 0.
git config --get-all http.https://tfs. myCie.com/defaultcollection/myProject/.extraheader
If I test this .ExtraHeader URL, it either doesn’t exist or I don’t have access !
git config --get-all http.proxy
git remote set-url origin https://emptyusername:***#tfs. myCie.com/defaultcollection/myProject/
fatal: unable to access 'https://tfs.myCie.com/defaultcollection/myProject/': OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to tfs.oecd.org:443
Can it be a proxy error or a user access account ?
In the later case, what access rights would I need ?
What settings I should use when creating the PAT token in TFS ?
Thanks.
I am afraid you were using the wrong serve connection type.
Azure Repos/Team Foundation Server service connection is used in the repositories resources section in the yaml pipeline, which you can refer to the repositories in other organizations using a service connection. See this document for more informaiton.
resources:
repositories:
- repository: otherrepo
name: ProjectName/RepoName
endpoint: newTFSServiceConnection
steps:
- checkout: self
- checkout: otherrepo
If you want to set up a pipeline for a repos in another TFS server. You need to create a new Service Connection of type Other Git
And enter the tfs repo url and users name / password in the edit page.
Then you can select this Other git type Service connection when creating a pipeline.
If you cannot connect to your tfs repo via PAT. It might be because the IIS Basic Authentication is enabled on your windows machine, it prevents you from using personal access tokens (PATs) as an authentication mechanism. See here, You can try using basic anthentication method (username and password) instead
Please be noted that you need to run this pipeline on your self-hosted agent. As you are trying to connect to the repo on your tfs intranet server. Cloud Microsoft hosted agent cannot connect to your intranet tfs server. Unless it is can be accessed in the public network.
I would like to deploy my Node.js app via GitHub to Azure.
I intend to make the app open source, thus no private info would be published in the repo; however, I still need to push the necessary credentials, API keys, etc. for the app to connect to other services.
How can I deploy the app without resorting to the private Git endpoint, and then awkward copy-pasting between the repos?
Typically you'll want to utilize an npm module like nconf to load environment variables from either a file or environment variables.
config.json is just a JSON document listing your key:value pairs. You'll want to add config.json to your .gitignore file to ensure you don't share your credentials publically.
Within the Azure Portal, you'll want to add your credentials as key:value pairs under Application Settings.
Note: You may be wondering what will happen if config.json is not found. nconf will simply move on to the next chained option. You could continue to chain config options together as in the following code snippet:
var nconf = require('nconf');
// Create nconf environtment
nconf
.file({ file: 'config.json' }) // Committed to repo; public settings
.file({file: 'local_config.json'}) // Not committed to repo; private or dev environment settings
.env();
Persistent data can be stored under d:\home, so I would recommend placing your private customizations there. If they need to be applied to the site in some way, you should do this by writing a deployment hook.
Set configuration as environment variables found in the "App Settings" section under Settings->Application Settings. Rationale here.
Your issue seems to be continuous deployment for Web App via Git from GitHub repo.
So I think #Dark Falcon 's answer is correct.
Azure continuous deployment support GitHub just need to do OAuth authentication in Azure Portal.
Find out the link "set up deployment from source control" at Azure WebApp Dashboard page and do it step by step, as the pictures below.
There is some blogs and vedio tutorials for details of helping you.
The blog explains how to use continuous deployment support for repo hosted on GitHub http://azure.microsoft.com/en-us/blog/using-app-service-web-apps-continuous-deployment-with-github-organizations/.
You also can follow these vedio tutorials to try to do it, as the references below.
http://azure.microsoft.com/en-us/documentation/videos/create-a-nodejs-site-deploy-from-github/
http://azure.microsoft.com/en-us/documentation/videos/deploying-to-azure-from-github/
https://channel9.msdn.com/Series/Windows-Azure-Web-Sites-Tutorials/Github-Continuous-Delivery-in-the-Preview-Portal.
Best Regards.