I have a process being executed in a Map/Reduce script which creates a bunch of quotes.
I would like to be able to add the Map/Reduce task ID to a field on the Quote/Estimate record.
The question therefore... is there a way to access the actual current task ID from within the M/R Script itself.
It would look something like: MAPREDUCETASK_02686f177c0a7667707763070b7c7752500068061c10016f1c470041_dc257c953b420bea6b7
Even though Task ID is not directly accessible from the script context (as it was already mentioned by ehcanadian), it can be obtained through Scheduled Script Instance search triggered from within the map/reduce script.
Example:
const _getCurrentTaskId = () => {
let scriptObj = runtime.getCurrentScript();
let scriptId = scriptObj.id;
let scriptDeploymentId = scriptObj.deploymentId;
let mapReduceTaskSearch = search.create({
type: search.Type.SCHEDULED_SCRIPT_INSTANCE,
filters: [
['status', 'anyof', 'PROCESSING'],
'AND',
['script.scriptid', 'is', scriptId],
'AND',
['scriptdeployment.scriptid', 'is', scriptDeploymentId]
],
columns: [
'taskid'
]
});
let taskId;
mapReduceTaskSearch.run().each(function(result) {
taskId = result.getValue('taskid');
});
return taskId;
}
Please note that if you have more SuiteCloud processors and running the same script deployment in parallel, this solution won't work (as it may return Task IDs of other currently running script deployments).
For MR you can get (and set) script ID and deployment ID, and ID of each task you ran from map/shuffle/reduce stages.
There isn't a method to get the currently executing M/R's task ID from within the M/R itself, as it's not exposed via the context. However, you could store that value in a custom record to be retrieved within the executing M/R if you're executing the M/R via script. (Checking script ID and deployment ID.) It won't be failsafe if you're executing more than one M/R of the same script though.
Related
Is it possible to easily see which roles have access to which script deployments?
I tried making a script deployments saved search as well as a role saved search but could not really find out how to extract this information.
Does anyone know?
I cannot see a way to complete this in the UI/via saved search. You can complete this via suitescript however by following this sort of layout. You can schedule the script to run periodically, or just run in the browser console to get the data you want. You can also add code to create an excel file to easily digest the information...
Layout in SS2.0
//gather all applicable role ids
var Roles = [];
//gather all applicable script deployment ids
var ScriptDep = [];
//for each script deployment id get the "Roles" from the "Audience" tab in the UI
for (var i=0; i<ScriptDep.length; i++){
var script = record.load({
type: 'scriptdeployment',
id: ScriptDep[i]
});
var scriptAudienceField = script.getField({
fieldId: 'audslctrole'
});
var scriptAudience = scriptAudienceField.getSelectOptions({
filter : '*',
operator : 'contains'
});
var RoleID = ; //role ID you care about, maybe loop through all roles with Roles[j]
var test = scriptAudience.includes(RoleID); //returns true or false this deployment is deployed to this role
}
Suite Answer 86327 gives the dollowing SS1.0 sample code
var search = nlapiLoadSearch('scriptdeployment', 147); //load search of all script deployments
var resultSet = search.runSearch();
resultSet.forEachResult(function(searchResult){ //for each script deployment returned by the search, get the id
var record = nlapiLoadRecord('scriptdeployment', searchResult.id); //load the script deployemnt
var arrayOfRoles = record.getFieldValues('audslctrole'); //get the values of the "Roles" from the "Audience" tab in the UI
if(arrayOfRoles == '18'){ //change based on the internal ID of the role
console.log(searchResult.id);
};
return true;
});
const query = `INSERT INTO countries VALUES (:country_id, :country_name)`;
try {
const result = await connection.execute(query, { country_id: 90, country_name: "Tonga" });
} catch (error) {
console.error(`error while executing: ${query}`);
}
Is there any way to log the query along with the bind parameters data
so that I can log INSERT INTO countries VALUES (90, "Tonga")
I think there's currently no builtin option to do that, but according to the docs you could create a wrapper around the execute function and log the actual query there. From the docs:
Sometimes it is useful to trace the bind data values that have been
used when executing statements. Several methods are available.
In the Oracle Database, the view V$SQL_BIND_CAPTURE can capture bind
information. Tracing with Oracle Database’s
DBMS_MONITOR.SESSION_TRACE_ENABLE() may also be useful.
You can also write your own wrapper around execute() and log any
parameters.
Eventually, I found a package known as bind-sql-string which has queryBindToString method that solves my problem. 🎉
I want to run more than one query .. how to do it?
eg, I have below two queries -
FOR doc IN users
RETURN doc
FOR doc IN users
RETURN { user: doc, newAttribute: true }
If I have to run both queries I have to run them separately, is there a way to execute a script or I need to put a semicolon at the end like SQL and run it.
Can I use arangosh?
You can use LET to execute multiple sub-queries in a single queries:
LET firstUserResult = (
FOR doc IN users
RETURN doc
)
LET secondUserResult = (
FOR doc IN users
RETURN { user: doc, newAttribute: true }
)
RETURN { first: firstUserResult, second: secondUserResult }
Some notes here - you will need to add an additional RETURN statement at the end of the query. This will definitely work for reads but you may run into issues when trying to write in multiple queries.
On Spark 2.0.x, I have been using a JobProgressListener implementation to retrieve Job/Stage/Task progress information in real-time from our cluster. I understand how the event flow works, and successfully receive updates on the work.
My problem is that we have several different submissions running at the same time on the same Spark Context, and it is seemingly impossible to differentiate between which Job/Stage/Task belongs to each submittal. Each Job/Stage/Task receives a unique id, which is great. However, I'm looking for a way to provide a submission "id" or "name" that would be returned along with the received JobProgressListener event objects.
I realize that the "Job Group" can be set on the Spark Context, but if multiple jobs are simultaneously running on the same context, they will become scrambled.
Is there a way I can sneak in custom properties that would be returned with the listener events for a single SQLContext? In so doing, I should be able to link up subsequent Stage and Task events and get what I need.
Please note: I am not using spark-submit for these jobs. They are being executed using Java references to a SparkSession/SQLContext.
Thanks for any solutions or ideas.
I'm using a local property - this can be accessed from listener during the onStageSubmit event. After that I'm using the corresponding stageId in order to identify the task executed during that stage.
Future({
sc.setLocalProperty("job-context", "second")
val listener = new MetricListener("second")
sc.addSparkListener(listener)
//do some spark actions
val df = spark.read.load("...")
val countResult = df.filter(....).count()
println(listener.rows)
sc.removeSparkListener(listener)
})
class MetricListener(name:String) extends SparkListener{
var rows: Long = 0L
var stageId = -1
override def onStageSubmitted(stageSubmitted: SparkListenerStageSubmitted): Unit = {
if (stageSubmitted.properties.getProperty("job-context") == name){
stageId = stageSubmitted.stageInfo.stageId
}
}
override def onTaskEnd(taskEnd: SparkListenerTaskEnd): Unit = {
if (taskEnd.stageId == stageId)
rows = rows + taskEnd.taskMetrics.inputMetrics.recordsRead
}
}
Why is no SQL being generated when I run my Nhibernate 3 query?
public IQueryable<Chapter> FindAllChapters()
{
using (ISession session = NHibernateHelper.OpenSession())
{
var chapters = session.QueryOver<Chapter>().List();
return chapters.AsQueryable();
}
}
If I run the query below I can see that the SQL that gets created.
public IQueryable<Chapter> FindAllChapters()
{
using (ISession session = NHibernateHelper.OpenSession())
{
var resultDTOs = session.CreateSQLQuery("SELECT Title FROM Chapter")
.AddScalar("Title", NHibernateUtil.String)
.List();
// Convert resultDTOs into IQueryable<Chapter>
}
}
Linq to NHibernate (like Linq to entities) uses delayed execution. You are returning IQueryable<Chapter> which means that you might add further filtering before using the data, so no query is executed.
If you called .ToList() or .List() (i forget which is in the API), then it would actually produce data and execute the query.
In other words, right now you have an unexecuted query.
Added: Also use Query() not QueryOver(). QueryOver is like detached criteria.
For more info, google "delayed execution linq" for articles like this