Azure Devops - run build only in the folder that have changes - azure

Question

I have a some spark jobs which are written on scala and build using maven.
Right now the follow a path like this
/src/job1/<<build data>>
/src/job2/<<build data>>
.....
This build data holds the code, pom.xml, tests and other thinks needed for each job pass to maven.
We have a root folder that has a father pom.xml that builds all the code, but I don't want to build all the code every time. I want to be able to test and build only the code for the jobs that changes.
The only way that I think is possible is to run build an tests only in the folders that changes was detected but I can't seem to do that without create one azure devops build for each folder, any ideas?
TL:DR - How to pass the path of folder that have changes in azure devops to the build process?

Answer 1

How to pass the path of folder that have changes in azure devops to the build process?
We could add a powershell scripts to check the folder or path, like:
$editedFiles = git diff HEAD HEAD~ --name-only
echo "$($editedFiles.Length) files modified:"
$editedFiles | ForEach-Object {
echo $_
Switch -Wildcard ($_ ) {
'/src/job1' {
Write-Output "##vso[task.setvariable variable=job1]True"
}
'/src/job2' {
Write-Output "##vso[task.setvariable variable=job2]True" }
}
}
This script sets variables which are then referenced in custom conditions in the pipeline build task or in the build pipeline:
and(succeeded(), eq(variables['job1'], 'True'))
You could check this similar thread for some details.
Hope this helps.

Answer 2

You can work around this by creating a script that will do the following:
Get the changeset/commit in the build to check which files are changed.
Based on the files changes, you define the logic to decide which folder it should build, run tests, etc.
This solution will require you to build all the logic to decide which folders to build and test.

Answer 3

Thanks for your inputs. I was able to do it with your inputs on git, I realized that we are using gitflow workflow and every time that we have pull request to the master branch (meaning deployment) a commit is created with starting "Merged PR", this way I can compare the latest two commits and check what changed.
How I'm doing:
# This part uses this to get the latest two pull requests hashes
$last_pr_commit=git log --grep="^Merged PR" -n 1 --pretty=format:"%h" --abbrev=200
$second_last_pr_commit=git log -n 1 --grep="^Merged PR" --oneline --abbrev=200 $last_pr_commit^...HEAD~2 --pretty=format:"%h"
# This get the path of the files
$result=git diff $last_pr_commit $second_last_pr_commit --name-only
And then I just build the ones that I'm interested.
ForEach ($line in $($result -split "`r`n"))
{
# In my case I only need the two first parts of path
$folderPath=$line.split('/')[0]+'/'+$line.split('/')[1]
if ($line.contains('.scala')){
# Running the tests and packaging
mvn package $folderPath
}
}
This is powershell but it can work with bash changing the for loop.
Thanks