How to call a function every interval time Odoo 11 - python-3.x

i know we can create automatic action using cron in odoo
but I want something a different
in the mass mailing of odoo i want to add a repetion option of mail mass mailings
Example in the Form view_mail_mass_mailing_form > Options page
I added a repetition selection field,
I added this because I want each mass mail alone
class MailMassMailing(models.Model):
_inherit = 'mail.mass_mailing'
recurrence_mail = fields.Selection([
('daily', 'Day'),
('weekly', 'Weeks'),
('monthly', 'Months'),
], string='Recurring')
I want this mass mailng to send each (days or weeks or months)
how to call a function with interval date,
how to call a function every (days or weeks or months)
The sending of this mass mailing is revived from the date of creation

Just extend Mass Mailing model with a new date field and implement a model method to use for a daily running ir.cron.
from odoo import api, fields, models
class MailMassMailing(models.Model):
_inherit = 'mail.mass_mailing'
recurrence_mail = fields.Selection([
('daily', 'Day'),
('weekly', 'Weeks'),
('monthly', 'Months'),
], string='Recurring')
last_sent_on = fields.Date()
#api.model
def run_send_recurring(self):
""" Resend mass mailing with recurring interval"""
domain = [('recurrence_mail', '!=', False)]
# TODO monthly should be solved in another way, but that
# is not needed for this example
deltas = {'daily': 1, 'weekly': 7, 'monthly': 30}
today = fields.Date.today()
for mass_mail in self.search(domain):
# never sent? go send it
if not mass_mail.last_sent_on:
# send the way you want
# or get delta between today and last_sent_on
last_dt = fields.Date.from_string(mass_mail.last_sent_on)
if (today - last_dt).days >= deltas[mass_mail.recurrence_mail]:
# send the way you want

Thank you #CZoellner for your help
I found the solution with your idea
# Solution ############### .py
#api.model
def run_send_recurring(self):
""" Resend mass mailing with recurring interval"""
date_format = '%Y-%m-%d'
domain = [('recurrence_mail', '!=', False),('state','=','done')]
deltas = {'daily': 1, 'weekly': 7, 'monthly': 30}
logger.info("______deltas________: %s ",deltas)
today = fields.Date.today()
logger.info("______today________: %s ",today)
for mass_mail in self.search(domain):
logger.info("______mass_mail________: %s ",mass_mail)
# never sent? go send it
if not mass_mail.last_sent_on:
self.put_in_queue()
joining_date = mass_mail.last_sent_on
current_date = (datetime.today()).strftime(date_format)
print('joining_date',joining_date)
d1 = datetime.strptime(joining_date, date_format).date()
logger.info("______1 day________: %s ",d1)
d2 = datetime.strptime(current_date, date_format).date()
logger.info("______2 day________: %s ",d2)
logger.info("______deltas[mass_mail.recurrence_mail]________: %s ",deltas[mass_mail.recurrence_mail])
r = relativedelta(d1,d2)
logger.info("______r day________: %s ",r.days)
if (r ,'>=' , deltas[mass_mail.recurrence_mail]):
mass_mail.put_in_queue()

Related

Odoo 13: Creating invoice from purchase order in odoo via api

I am brand new to odoo! On odoo 13 EE I am trying to create and confirm a vendor bill after importing a purchase order and the item receipts. I can create an invoice directly, but haven't been able to link that to the PO/receipt?
Sadly under purchase.order the method action_create_invoice seems hidden from the API
order_id = PurchaseOrder.create(po)
purchaseorder = PurchaseOrder.browse([order_id])
print("Before validating:", purchaseorder.name, purchaseorder.state) # draft
odoo.env.context['check_move_validity'] = True
purchaseorder.button_confirm()
purchaseorder = PurchaseOrder.browse([order_id])
picking_count = purchaseorder.picking_count
print("After Post:", purchaseorder.name, purchaseorder.state, "picking_count = ", purchaseorder.picking_count)
if picking_count == 0:
print("Nothing to receive. Straight to to Billing.") # ok so far
tryme = purchaseorder.action_view_invoice()
## Error => odoorpc.error.RPCError: type object 'purchase.order' has no attribute 'action_create_invoice'
SO I tried overriding/extending this way
class PurchaseOrder(models.Model):
_inherit = 'purchase.order'
#api.model
def create_invoice(self, context=None):
# try 1 => odoorpc.error.RPCError: 'super' object has no attribute # 'action_create_invoice'
rtn = super().action_create_invoice(self)
# try2 => odoorpc.error.RPCError: name 'action_create_invoice' is # not defined
# rtn = action_create_invoice(self)
# try3 => Error %s 'super' object has no attribute ' # action_create_invoice'
# rtn = super(models.Model, self).action_create_invoice(self)
return rtn
I hope somebody can suggest a solution! Thank you.
Please dont customize it without having a functional knowledge in odoo. In odoo, if you go to purchase settings, you can find billing options under invoicing where you can find 2 options, ordered quantity and received quantity. if it is ordered quantity, then you can create invoice after confirming the Purchase order. if it is received quantity, then after confirming the purchase order, a incoming shipment will be created and after the incoming shipment is processed, you can find the create invoice button in purchase order
If you can do it from the browser client, than you should just look what API commands the browser sends to the odoo server (in Chrome by enabling the debug view by pressing F12, and looking in the network tab), so that you just need to copy that communication.

slider.value values not getting updated using ColumnDataSource(Dataframe).data

I have been working on COVID19 analysis for a dashboard and am using a JSON data source. I have converted the json to dataframe. I am working on plotting bar chart for "Days to reach deaths" over a "States" x-axis (categorical values). I am trying to use a function to update the slider.value. Upon running the bokeh serve with --log-level=DEBUG, I am getting a following error:
Can someone provide me with any direction or help with what might be causing the issue as I am new to Python and any help is appreciated? Or if there's any other alternative.
Please find the code below:
cases_summary = requests.get('https://api.rootnet.in/covid19-in/stats/history')
json_data = cases_summary.json()
#Data Cleaning
cases_summary=pd.json_normalize(json_data['data'], record_path='regional', meta='day')
cases_summary['loc']=np.where(cases_summary['loc']=='Nagaland#', 'Nagaland', cases_summary['loc'])
cases_summary['loc']=np.where(cases_summary['loc']=='Madhya Pradesh#', 'Madhya Pradesh', cases_summary['loc'])
cases_summary['loc']=np.where(cases_summary['loc']=='Jharkhand#', 'Jharkhand', cases_summary['loc'])
#Calculate cumulative days since 1st case for each state
cases_summary['day_count']=(cases_summary['day'].groupby(cases_summary['loc']).cumcount())+1
#Initial plot for default slider value=35
days_reach_death_count=cases_summary.loc[(cases_summary['deaths']>=35)].groupby(cases_summary['loc']).head(1).reset_index()
slider = Slider(start=10, end=max(cases_summary['deaths']), value=35, step=10, title="Total Deaths")
source = ColumnDataSource(data=dict(days_reach_death_count[['loc','day_count', 'deaths']]))
q = figure(x_range=days_reach_death_count['loc'], plot_width=1200, plot_height=600, sizing_mode="scale_both")
q.title.align = 'center'
q.title.text_font_size = '17px'
q.xaxis.axis_label = 'State'
q.yaxis.axis_label = 'Days since 1st Case'
q.xaxis.major_label_orientation = math.pi/2
q.vbar('loc', top='day_count', width=0.9, source=source)
deaths = slider.value
q.title.text = 'Days to reach %d Deaths' % deaths
hover = HoverTool(line_policy='next')
hover.tooltips = [('State', '#loc'),
('Days since 1st Case', '#day_count'), # #$name gives the value corresponding to the legend
('Deaths', '#deaths')
]
q.add_tools(hover)
def update(attr, old, new):
days_death_count = cases_summary.loc[(cases_summary['deaths'] >= slider.value)].groupby(cases_summary['loc']).head(1).reindex()
source.data = [ColumnDataSource().from_df(days_death_count)]
slider.on_change('value', update)
layout = row(q, slider)
tab = Panel(child=layout, title="New Confirmed Cases since Day 1")
tabs= Tabs(tabs=[tab])
curdoc().add_root(tabs)
Your code has 2 issues
(critical) source.data must be a dictionary, but you're assigning it an array
(minor) from_df is a class method, you don't have to construct an object of it
Try using source.data = ColumnDataSource.from_df(days_death_count) instead.

Telegram Bot profile with calendar

I am writing a bot profile. After the answer, the following question should be asked. In one of the steps you need to select a date. I use a ready-made calendar. But after choosing a date, the next question is not asked, but the script does not crash, only waits for input and throws the Exception defined in the code below. How to fix that would bring up the next question "What is your name?". Thank. Here is the part of the code that includes the previous calendar question and the next question:
def process_Personenanzahl_step(message):
try:
chat_id = message.chat.id
user = user_dict[chat_id]
user.carModel = message.text
markup = types.ReplyKeyboardMarkup(one_time_keyboard=True, resize_keyboard=True)
itembtn1 = types.KeyboardButton('1')
itembtn2 = types.KeyboardButton('2')
itembtn3 = types.KeyboardButton('3')
itembtn4 = types.KeyboardButton('4')
itembtn5 = types.KeyboardButton('5')
markup.add(itembtn1, itembtn2, itembtn3, itembtn4, itembtn5)
msg = bot.send_message(chat_id, 'Qun', reply_markup=markup)
bot.register_next_step_handler(msg, check_calendar_messages)
except Exception as e:
bot.reply_to(message, 'ooops!!')
calendar_1 = CallbackData("calendar_1", "action", "year", "month", "day")
def check_calendar_messages(message):
now = datetime.datetime.now() # Get the current date
bot.send_message(
message.chat.id,
"Selected date",
reply_markup=telebot_calendar.create_calendar(
name=calendar_1.prefix,
year=now.year,
month=now.month, # Specify the NAME of your calendar
),
)
#bot.callback_query_handler(func=lambda call: call.data.startswith(calendar_1.prefix))
def callback_inline(call: object) -> object:
# At this point, we are sure that this calendar is ours. So we cut the line by the separator of our calendar
name, action, year, month, day = call.data.split(calendar_1.sep)
# Processing the calendar. Get either the date or None if the buttons are of a different type
date = telebot_calendar.calendar_query_handler(
bot=bot, call=call, name=name, action=action, year=year, month=month, day=day
)
# There are additional steps. Let's say if the date DAY is selected, you can execute your code. I sent a message.
if action == "DAY":
msg = bot.send_message(
chat_id=call.from_user.id,
text=f"You have chosen {date.strftime('%d.%m.%Y')}",
reply_markup=ReplyKeyboardRemove(),
)
print(f"{calendar_1}: Day: {date.strftime('%d.%m.%Y')}")
bot.register_next_step_handler(msg, process_fullname_step)
def process_fullname_step(message):
try:
chat_id = message.chat.id
user_dict[chat_id] = User(message.text)
markup = types.ReplyKeyboardRemove(selective=False)
msg = bot.send_message(chat_id, 'What is your name?', reply_markup=markup)
bot.register_next_step_handler(msg, process_fullname_step)
except Exception as e:
bot.reply_to(message, 'ooops!!')

check availability of meeting rooms in outlook with python

I am working on to write a python script to check if particular meeting room is available. If yes then meeting room will be booked, if not then python will find available time slot for that day.
For now, I have achieved to book meeting room but I am not able to check availability of rooms.
To book any meeting room, i have to send mail to that book meeting room configured mail id and corresponding acceptance/decline mail I receive as per the availability.
below is the snippet :
import win32com.client
import datetime
import pywintypes
oOutlook = win32com.client.Dispatch("Outlook.Application")
appt = oOutlook.CreateItem(1)
appt.Start = '2018-05-18 13:30'
appt.Subject = 'Follow Up Meeting'
appt.Duration = 30
appt.Location = '<name of meeting room>'
appt.MeetingStatus = 1
myRecipient = appt.Recipients.Add("<mail id of meeting room")
myRecipient.resolve
my_date = datetime.date(2018,5,18)
pywintypeDate = pywintypes.Time (my_date)
availabilityInfo = myRecipient.FreeBusy(pywintypeDate,30,True)
print(availabilityInfo)
# appt.Save()
# appt.Send()
# print("done")
output is :
000000000000000000000222222200222222022000000000000000000000000000000002222222222220000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022000000000000000000000000000000000000000002220002222200000000000000000000000000000000002220022022222000000000000000000000000000000000000000000002222000000000000000000000000000000000000220000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022000000000000000000000000000000000000000000000222222200000000000000000000000000000000002220000022000000000000000000000000000000000000002220000222222000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022000000000000000000000000000000000000000002220022022200000000000000000000000000000000000022000022000000000000000000000000000000000000000000000002222000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022000000000000000000000000000000000000000000000022022200000000000000000000000000000000002220002222000000000000000
so is it first byte (0) indicate time slot from 00:00 to 00:30 and soon for one complete month ?
Is it possible to get output only for one day ?
Do i have to parse the above output to check availability for my particular required time ?
appt.Recipients.Add returns the Recipient object. Resolve it first (Recipient.Resolve), then call Recipient.FreeBusy.

Exporting from MS Excel to MS Access with intermediate processing

I have an application which produces reports in Excel (.XLS) format. I need to append the data from these reports to an existing table in a MS Access 2010 database. A typical record is:
INC000000004154 Closed Cbeebies BBC Childrens HQ6 monitor wall dropping out. HQ6 P3 3/7/2013 7:03:01 PM 3/7/2013 7:03:01 PM 3/7/2013 7:14:15 PM The root cause of the problem was the power supply to the PC which was feeding the monitor. HQ6 Monitor wall dropping out. BBC Third Party Contractor supply this equipment.
The complication is that I need to do some limited processing on the data. Viz
Specifically I need to do a couple of lookups converting names to numbers and also parse a date-string (the report for some reason puts the dates in to the spreadsheet in text format rather than date format).
Now I could do this in Python using XLRD/XLWT but would much prefer to do it in Excel or Access. Does anyone have any advice on a good way to approach this? I would very much prefer NOT to use VBA so could I do something like record an MS Excel macro and then execute that macro on the newly created XLS file?
You can directly import some Excel data into MS Access, but if your requirement is to do some processing because then I don't see how you will be able to achieve that without:
an ETL application, like Pentaho or Talend or others.
That will certainly be like using a hammer to crush an ant though.
some other external data processing pipeline, in Python or some other programming language.
VBA (wether through macros or hand coded).
VBA has been really good at doing that sort of things in Access for literally decades.
Since you are using Excel and Access, staying within that realm looks like the best solution for solving your issue.
Just use queries:
You import the data without transformation to a table whose sole purpose is to accommodate the data from Excel; then you create queries from that raw data to add the missing information and massage the data before appending the result into your final destination table.
That solution has the advantage of letting you create simple steps in Access that you can easily record using macros.
I asked this question some time ago and decided it would be easier to do it in Python. Gord asked me to share, and here it is (sorry about the delay, other projects took priority for a while).
"""
Routine to migrate the S7 data from MySQL to the new Access
database.
We're using the pyodbc libraries to connect to Microsoft Access
Note that there are 32- and 64-bit versions of these libraries
available but in order to work the word-length for pyodbc and by
implication Python and all its associated compiled libraries must
match that of MS Access. Which is an arse as I've just had to
delete my 64-bit installation of Python and replace it and all
the libraries with the 32-bit version.
Tim Greening-Jackson 08 May 2013 (timATgreening-jackson.com)
"""
import pyodbc
import re
import datetime
import tkFileDialog
from Tkinter import *
class S7Incident:
"""
Class containing the records downloaded from the S7.INCIDENTS table
"""
def __init__(self, id_incident, priority, begin, acknowledge,
diagnose, workaround,fix, handoff, lro, nlro,
facility, ctas, summary, raised, code):
self.id_incident=unicode(id_incident)
self.priority = {u'P1':1, u'P2':2, u'P3':3, u'P4':4, u'P5':5} [unicode(priority.upper())]
self.begin = begin
self.acknowledge = acknowledge
self.diagnose = diagnose
self.workaround = workaround
self.fix = fix
self.handoff = True if handoff else False
self.lro = True if lro else False
self.nlro = True if nlro else False
self.facility = unicode(facility)
self.ctas = ctas
self.summary = "** NONE ***" if type(summary) is NoneType else summary.replace("'","")
self.raised = raised.replace("'","")
self.code = 0 if code is None else code
self.production = None
self.dbid = None
def __repr__(self):
return "[{}] ID:{} P{} Prod:{} Begin:{} A:{} D:+{}s W:+{}s F:+{}s\nH/O:{} LRO:{} NLRO:{} Facility={} CTAS={}\nSummary:'{}',Raised:'{}',Code:{}".format(
self.id_incident,self.dbid, self.priority, self.production, self.begin,
self.acknowledge, self.diagnose, self.workaround, self.fix,
self.handoff, self.lro, self.nlro, self.facility, self.ctas,
self.summary, self.raised, self.code)
def ProcessIncident(self, cursor, facilities, productions):
"""
Produces the SQL necessary to insert the incident in to the Access
database, executes it and then gets the autonumber ID (dbid) of the newly
created incident (this is used so LRO, NRLO CTAS and AD1 can refer to
their parent incident.
If the incident is classed as LRO, NLRO, CTAS then the appropriate
record is created. Returns the dbid.
"""
if self.raised.upper() in productions:
self.production = productions[self.raised.upper()]
else:
self.production = 0
sql="""INSERT INTO INCIDENTS (ID_INCIDENT, PRIORITY, FACILITY, BEGIN,
ACKNOWLEDGE, DIAGNOSE, WORKAROUND, FIX, HANDOFF, SUMMARY, RAISED, CODE, PRODUCTION)
VALUES ('{}', {}, {}, #{}#, {}, {}, {}, {}, {}, '{}', '{}', {}, {})
""".format(self.id_incident, self.priority, facilities[self.facility], self.begin,
self.acknowledge, self.diagnose, self.workaround, self.fix,
self.handoff, self.summary, self.raised, self.code, self.production)
cursor.execute(sql)
cursor.execute("SELECT ##IDENTITY")
self.dbid = cursor.fetchone()[0]
if self.lro:
self.ProcessLRO(cursor, facilities[self.facility])
if self.nlro:
self.ProcessNLRO(cursor, facilities[self.facility])
if self.ctas:
self.ProcessCTAS(cursor, facilities[self.facility], self.ctas)
return self.dbid
def ProcessLRO(self, cursor, facility):
sql = "INSERT INTO LRO (PID, DURATION, FACILITY) VALUES ({}, {}, {})"\
.format(self.dbid, self.workaround, facility)
cursor.execute(sql)
def ProcessNLRO(self, cursor, facility):
sql = "INSERT INTO NLRO (PID, DURATION, FACILITY) VALUES ({}, {}, {})"\
.format(self.dbid, self.workaround, facility)
cursor.execute(sql)
def ProcessCTAS(self, cursor, facility, code):
sql = "INSERT INTO CTAS (PID, DURATION, FACILITY, CODE) VALUES ({}, {}, {}, {})"\
.format(self.dbid, self.workaround, facility, self.ctas)
cursor.execute(sql)
class S7AD1:
"""
S7.AD1 records.
"""
def __init__(self, id_ad1, date, ref, commentary, adjustment):
self.id_ad1 = id_ad1
self.date = date
self.ref = unicode(ref)
self.commentary = unicode(commentary)
self.adjustment = float(adjustment)
self.pid = 0
self.production = 0
def __repr__(self):
return "[{}] Date:{} Parent:{} PID:{} Amount:{} Commentary: {} "\
.format(self.id_ad1, self.date.strftime("%d/%m/%y"), self.ref, self.pid, self.adjustment, self.commentary)
def SetPID(self, pid):
self.pid = pid
def SetProduction(self, p):
self.production = p
def Process(self, cursor):
sql = "INSERT INTO AD1 (pid, begin, commentary, production, adjustment) VALUES ({}, #{}#, '{}', {}, {})"\
.format(self.pid, self.date.strftime("%d/%m/%y"), self.commentary, self.production, self.adjustment)
cursor.execute(sql)
class S7Financial:
"""
S7 monthly financial summary of income and penalties from S7.FINANCIALS table.
These are identical in the new database
"""
def __init__(self, month, year, gco, cta, support, sc1, sc2, sc3, ad1):
self.begin = datetime.date(year, month, 1)
self.gco = float(gco)
self.cta = float(cta)
self.support = float(support)
self.sc1 = float(sc1)
self.sc2 = float(sc2)
self.sc3 = float(sc3)
self.ad1 = float(ad1)
def __repr__(self):
return "Period: {} GCO:{:.2f} CTA:{:.2f} SUP:{:.2f} SC1:{:.2f} SC2:{:.2f} SC3:{:.2f} AD1:{:.2f}"\
.format(self.start.strftime("%m/%y"), self.gco, self.cta, self.support, self.sc1, self.sc2, self.sc3, self.ad1)
def Process(self, cursor):
"""
Insert in to FINANCIALS table
"""
sql = "INSERT INTO FINANCIALS (BEGIN, GCO, CTA, SUPPORT, SC1, SC2, SC3, AD1) VALUES (#{}#, {}, {}, {}, {}, {}, {},{})"\
.format(self.begin, self.gco, self.cta, self.support, self.sc1, self.sc2, self.sc3, self.ad1)
cursor.execute(sql)
class S7SC3:
"""
Miscellaneous S7 SC3 stuff. The new table is identical to the old one.
"""
def __init__(self, begin, month, year, p1ot, p2ot, totchg, succchg, chgwithinc, fldchg, egychg):
self.begin = begin
self.p1ot = p1ot
self.p2ot = p2ot
self.changes = totchg
self.successful = succchg
self.incidents = chgwithinc
self.failed = fldchg
self.emergency = egychg
def __repr__(self):
return "{} P1:{} P2:{} CHG:{} SUC:{} INC:{} FLD:{} EGY:{}"\
.format(self.period.strftime("%m/%y"), self.p1ot, self.p1ot, self.changes, self.successful, self.incidents, self.failed, self.emergency)
def Process(self, cursor):
"""
Inserts a record in to the Access database
"""
sql = "INSERT INTO SC3 (BEGIN, P1OT, P2OT, CHANGES, SUCCESSFUL, INCIDENTS, FAILED, EMERGENCY) VALUES\
(#{}#, {}, {}, {}, {}, {}, {}, {})"\
.format(self.begin, self.p1ot, self.p2ot, self.changes, self.successful, self.incidents, self.failed, self.emergency)
cursor.execute(sql)
def ConnectToAccessFile():
"""
Prompts the user for an Access database file, connects, creates a cursor,
cleans out the tables which are to be replaced, gets a hash of the facilities
table keyed on facility name returning facility id
"""
# Prompts the user to select which Access DB file he wants to use and then attempts to connect
root = Tk()
dbname = tkFileDialog.askopenfilename(parent=root, title="Select output database", filetypes=[('Access 2010', '*.accdb')])
root.destroy()
# Connect to the Access (new) database and clean its existing incidents etc. tables out as
# these will be replaced with the new data
dbcxn = pyodbc.connect("Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ="+dbname+";")
dbcursor=dbcxn.cursor()
print("Connected to {}".format(dbname))
for table in ["INCIDENTS", "AD1", "LRO", "NLRO", "CTAS", "SC3", "PRODUCTIONS", "FINANCIALS"]:
print("Clearing table {}...".format(table))
dbcursor.execute("DELETE * FROM {}".format(table))
# Get the list of facilities from the Access database...
dbcursor.execute("SELECT id, facility FROM facilities")
rows = dbcursor.fetchall()
dbfacilities = {unicode(row[1]):row[0] for row in rows}
return dbcxn, dbcursor, dbfacilities
# Entry point
incre = re.compile("INC\d{12}[A-Z]?") # Regex that matches incident references
try:
dbcxn, dbcursor, dbfacilities = ConnectToAccessFile()
# Connect to the MySQL S7 (old) database and read the incidents and ad1 tables
s7cxn = pyodbc.connect("DRIVER={MySQL ODBC 3.51 Driver}; SERVER=localhost;DATABASE=s7; UID=root; PASSWORD=********; OPTION=3")
print("Connected to MySQL S7 database")
s7cursor = s7cxn.cursor()
s7cursor.execute("""
SELECT id_incident, priority, begin, acknowledge,
diagnose, workaround, fix, handoff, lro, nlro,
facility, ctas, summary, raised, code FROM INCIDENTS""")
rows = s7cursor.fetchall()
# Discard any incidents which don't have a reference of the form INC... as they are ancient
print("Fetching incidents")
s7incidents = {unicode(row[0]):S7Incident(*row) for row in rows if incre.match(row[0])}
# Get the list of productions from the S7 database to replace the one we've just deleted ...
print("Fetching productions")
s7cursor.execute("SELECT DISTINCT RAISED FROM INCIDENTS")
rows = s7cursor.fetchall()
s7productions = [r[0] for r in rows]
# ... now get the AD1s ...
print("Fetching AD1s")
s7cursor.execute("SELECT id_ad1, date, ref, commentary, adjustment from AD1")
rows = s7cursor.fetchall()
s7ad1s = [S7AD1(*row) for row in rows]
# ... and the financial records ...
print("Fetching Financials")
s7cursor.execute("SELECT month, year, gco, cta, support, sc1, sc2, sc3, ad1 FROM Financials")
rows = s7cursor.fetchall()
s7financials = [S7Financial(*row) for row in rows]
print("Writing financials ({})".format(len(s7financials)))
[p.Process(dbcursor) for p in s7financials]
# ... and the SC3s.
print("Fetching SC3s")
s7cursor.execute("SELECT begin, month, year, p1ot, p2ot, totchg, succhg, chgwithinc, fldchg, egcychg from SC3")
rows = s7cursor.fetchall()
s7sc3s = [S7SC3(*row) for row in rows]
print("Writing SC3s ({})".format(len(s7sc3s)))
[p.Process(dbcursor) for p in s7sc3s]
# Re-create the productions table in the new database. Note we refer to production
# by number in the incidents table so need to do the SELECT ##IDENTITY to give us the
# autonumber index. To make sure everything is case-insensitive convert the
# hash keys to UPPERCASE.
dbproductions = {}
print("Writing productions ({})".format(len(s7productions)))
for p in sorted(s7productions):
dbcursor.execute("INSERT INTO PRODUCTIONS (PRODUCTION) VALUES ('{}')".format(p))
dbcursor.execute("SELECT ##IDENTITY")
dbproductions[p.upper()] = dbcursor.fetchone()[0]
# Now process the incidents etc. that we have retrieved from the S7 database
print("Writing incidents ({})".format(len(s7incidents)))
[s7incidents[k].ProcessIncident(dbcursor, dbfacilities, dbproductions) for k in sorted(s7incidents)]
# Match the new parent incident IDs in the AD1s and then write to the new table. Some
# really old AD1s don't have the parent incident reference in the REF field, it is just
# mentioned SOMEWHERE in the commentary. So if the REF field doesn't match then do a
# re.search (not re.match!) for it. It isn't essential to match these older AD1s with
# their parent incident, but it is quite useful (and tidy).
print("Matching and writing AD1s".format(len(s7ad1s)))
for a in s7ad1s:
if a.ref in s7incidents:
a.SetPID(s7incidents[a.ref].dbid)
a.SetProduction(s7incidents[a.ref].production)
else:
z=incre.search(a.commentary)
if z and z.group() in s7incidents:
a.SetPID(s7incidents[z.group()].dbid)
a.SetProduction(s7incidents[z.group()].production)
a.Process(dbcursor)
print("Comitting changes")
dbcursor.commit()
finally:
print("Closing databases")
dbcxn.close()
s7cxn.close()
It turns out that the file has additional complications in terms of mangled data which will require a degree of processing which is a pain to do in Excel but trivially simple in Python. So I will re-use some Python 2.x scripts which use the XLWT/XLRD libraries to munge the spreadsheet.

Resources