I have created an office script (cannot use VBA, because it has to work in Excel online), which transforms data and converts it to table. However, it only works only if I have no more that 4 rows of data, if there are more rows the rest does not get processed fully, seems like the script runs out of time and runs last lines of code or something.
Source data table (in sheet2) looks like this:
Code
Column1
Column2
Column3
Column4
Column5
Column6
Column7
Column8
Column9
Column10
Column11
PowerAppsId
a
1
2
3
4
5
6
7
8
9
10
11
12
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 315155Pcs = 19Mtr = 228.00Cor = Sor = Btc = 21KE020558
1d4dfa2c-ed94-4152-adcd-6700e208608f
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 36829EPcs = 19Mtr = 228Cor = Sor = Btc = Z000470271
c7a27d77-36bd-407e-8131-7180d753b79f
Dia = 219,1Thk = 12,5Len = 12000Qua = S460MHHea = 032111Pcs = 4Mtr = 48Sor = STOCKCor = Btc = 21C1010406
ded5157e-f82f-46b1-897a-1f2874f40900
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 36829EPcs = 19Mtr = 228Cor = Sor = Btc = Z000470273
5fd82d95-a416-4d1f-ab0f-bee896a5d5e2
Dia = 219,1Thk = 12,5Len = 12000Qua = S460MHHea = 032111Pcs = 4Mtr = 48Sor = STOCKCor = Btc = 21C1010406
743fd8c3-8e79-4d72-86af-f46f50a44644
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 36829EPcs = 19Mtr = 228Cor = Sor = Btc = Z000470271
8bb45032-cab4-452d-a044-c189461967a7
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 315155Pcs = 19Mtr = 228.00Cor = Sor = Btc = 21KE020558
e7e3cddd-0ae2-40a1-bbc0-1191817f8f02
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 36829EPcs = 19Mtr = 228Cor = Sor = Btc = Z000470271
25347ead-8ccf-43e4-a8f0-b34517dc8526
Dia = 219,1Thk = 12,5Len = 12000Qua = S460MHHea = 032111Pcs = 4Mtr = 48Sor = STOCKCor = Btc = 21C1010406
852ff0b5-41ef-42ba-afa0-18cb7b83bf80
Dia = 88.90Thk = 6.30Len = 12000Qua = S460MHHea = 36829EPcs = 19Mtr = 228Cor = Sor = Btc = Z000470273
a45eada1-b926-48bf-872b-2f5a8150a5b1
Then I run the script:
function main(workbook: ExcelScript.Workbook) {
let selectedSheet = workbook.getWorksheet("Sheet2");
// Replace all "Dia = " with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Dia = ", "", {completeMatch: false, matchCase: false});
// Replace all "Thk =" with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Thk =", "", {completeMatch: false, matchCase: false});
// Replace all "Len =" with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Len =", "", {completeMatch: false, matchCase: false});
// Replace all "Qua =" with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Qua =", "", {completeMatch: false, matchCase: false});
// Replace all "Hea =" with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Hea =", "", {completeMatch: false, matchCase: false});
// Replace all "Pcs =" with "" on extended range obtained by extending down from range A3 on selectedSheet
selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).replaceAll("Pcs =", "", {completeMatch: false, matchCase: false});
// Replace all "Sor = " with " SOR" on range A3:A6 on selectedSheet
selectedSheet.getRange("A3:A6").replaceAll("Sor = ", " SOR", {completeMatch: false, matchCase: false});
// Replace all "Cor = " with "" on range A3:A6 on selectedSheet
selectedSheet.getRange("A3:A6").replaceAll("Cor = ", "", {completeMatch: false, matchCase: false});
// Replace all "Btc =" with "" on range A3:A6 on selectedSheet
selectedSheet.getRange("A3:A6").replaceAll("Btc =", "", {completeMatch: false, matchCase: false});
// Replace all "Mtr =" with "" on range A3:A6 on selectedSheet
selectedSheet.getRange("A3:A6").replaceAll("Mtr =", "", {completeMatch: false, matchCase: false});
// Text to columns on range A3:A6 on selectedSheet
for (let row = 0; row < selectedSheet.getRange("A3:A6").getRowCount() ; row++) {
let sourceRange = selectedSheet.getRange("A3:A6");
let destinationRange = selectedSheet.getRange("A3");
let sourceRangeValues = sourceRange.getRow(row).getValues()[0][0].toString().split(/[ ]/)
destinationRange.getOffsetRange(row, 0).getResizedRange(0, sourceRangeValues.length - 1).setValues([sourceRangeValues]);
}
let sheet1 = workbook.getWorksheet("Sheet1");
// Delete extended range obtained by extending down from range 2:2 on selectedSheet
sheet1.getRange("2:2").getExtendedRange(ExcelScript.KeyboardDirection.down).delete(ExcelScript.DeleteShiftDirection.up);
let table1 = workbook.getTable("Table1");
// Paste to table table1 cell in row 0 on column Dia from extended range obtained by extending down, then right from range A3 on selectedSheet
table1.getColumn("Dia").getRangeBetweenHeaderAndTotal().getRow(0).copyFrom(selectedSheet.getRange("A3").getExtendedRange(ExcelScript.KeyboardDirection.down).getExtendedRange(ExcelScript.KeyboardDirection.right), ExcelScript.RangeCopyType.all, false, false);
selectedSheet.getRange("3:3").getExtendedRange(ExcelScript.KeyboardDirection.down).delete(ExcelScript.DeleteShiftDirection.up);
}
but it outputs table like this (to sheet1):
Dia
Thk
Len
Qua
Hea
Pcs
Mtr
Cor
Btc
88.90
6.30
12000
S460MH
315155
19
228.00
SOR
21KE020558
88.90
6.30
12000
S460MH
36829E
19
228
SOR
Z000470271
219,1
12,5
12000
S460MH
032111
4
48
SORSTOCK
21C1010406
88.90
6.30
12000
S460MH
36829E
19
228
SOR
Z000470273
219,1 12,5 12000 S460MH 032111 4Mtr = 48Sor = STOCKCor = Btc = 21C1010406
88.90 6.30 12000 S460MH 36829E 19Mtr = 228Cor = Sor = Btc = Z000470271
88.90 6.30 12000 S460MH 315155 19Mtr = 228.00Cor = Sor = Btc = 21KE0205583
88.90 6.30 12000 S460MH 36829E 19Mtr = 228Cor = Sor = Btc = Z000470271
219,1 12,5 12000 S460MH 032111 4Mtr = 48Sor = STOCKCor = Btc = 21C1010406
88.90 6.30 12000 S460MH 36829E 19Mtr = 228Cor = Sor = Btc = Z000470273
As you can see, first four rows are processed correctly, but the rest only partially. What can be causing this? Script shows no errors
I have tried running the same script locally - saved excel online file as a local copy, but results are the same. Script works normally only if I have up to four rows of data
Range was not defined correctly in part of the script
Related
I am trying to write data from Strava to a treeview, however, when it writes I will get the error
_tkinter.TclError: character U+1f1e8 is above the range (U+0000-U+FFFF) allowed by Tcl
I realised this was due to someone using an emoji in the name of their ride. I wrote code that only skips some Unicode values and not others. My code is seen below:
count = 0
distanceM = 0
distanceML = 0
elevationM = 0
elevationFT = 0
full_name = ""
for row in strava.get_children():
strava.delete(row)
auth_url = "https://www.strava.com/oauth/token"
activites_url = "https://www.strava.com/api/v3/clubs/2885/activities"
payload = {
'client_id': "XXXXX",
'client_secret': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
'refresh_token': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
'grant_type': "refresh_token",
'f': 'json'
}
res = requests.post(auth_url, data=payload, verify=False)
access_token = res.json()['access_token']
ride_name = ""
header = {'Authorization': 'Bearer ' + access_token}
param = {'per_page':200, 'page':1}
my_dataset = requests.get(activites_url, headers=header, params=param).json()
for i in range (0,len(my_dataset)):
entry = 1
strava.tag_configure('odd',background = "#fc4d03")
strava.tag_configure('even', background = 'white')
full_name = str(my_dataset[i]["athlete"]["firstname"])+ " " + str(my_dataset[i]["athlete"]["lastname"])
distanceM = int(my_dataset[i]["distance"])
distanceML = distanceM * 0.000621371
distanceML = '%.2f'%(distanceML)
elevationM = int(my_dataset[i]["total_elevation_gain"])
elevationFT = elevationM * 3.28084
elevationFT = '%.0f'%(elevationFT)
ride_name = (my_dataset[i]["name"])
try:
ride_name.encode('ascii')
except UnicodeEncodeError:
entry = 0
if entry == 1:
if count%2 == 0:
strava.insert("", "end", text="", values=(full_name,my_dataset[i]["name"],distanceML,elevationFT), tag = ('even',))
else:
strava.insert("", "end", text="", values=(full_name,my_dataset[i]["name"],distanceML,elevationFT), tag = ('odd',))
count = count+1
How I can get actual coordinates of Landsat image corners (see image to understand) ?
From metadata file (..._MTL.txt) I can get coordinates of red corners, but I need to get coordinates of green corners.
I work with GeoTIFF files using GDAL.
I need to get correct latitude and longitude of green points.
Can I do it using python3?
Thanks for help
Metadata file
GROUP = L1_METADATA_FILE
GROUP = METADATA_FILE_INFO
ORIGIN = "Image courtesy of the U.S. Geological Survey"
REQUEST_ID = "9991103150002_00325"
PRODUCT_CREATION_TIME = 2011-03-16T20:14:24Z
STATION_ID = "EDC"
LANDSAT5_XBAND = "1"
GROUND_STATION = "IKR"
LPS_PROCESSOR_NUMBER = 0
DATEHOUR_CONTACT_PERIOD = "1016604"
SUBINTERVAL_NUMBER = "01"
END_GROUP = METADATA_FILE_INFO
GROUP = PRODUCT_METADATA
PRODUCT_TYPE = "L1T"
ELEVATION_SOURCE = "GLS2000"
PROCESSING_SOFTWARE = "LPGS_11.3.0"
EPHEMERIS_TYPE = "DEFINITIVE"
SPACECRAFT_ID = "Landsat5"
SENSOR_ID = "TM"
SENSOR_MODE = "BUMPER"
ACQUISITION_DATE = 2010-06-15
SCENE_CENTER_SCAN_TIME = 04:57:44.2830500Z
WRS_PATH = 145
STARTING_ROW = 26
ENDING_ROW = 26
BAND_COMBINATION = "1234567"
PRODUCT_UL_CORNER_LAT = 49.8314223
PRODUCT_UL_CORNER_LON = 84.0018859
PRODUCT_UR_CORNER_LAT = 49.8694055
PRODUCT_UR_CORNER_LON = 87.4313889
PRODUCT_LL_CORNER_LAT = 47.8261840
PRODUCT_LL_CORNER_LON = 84.1192898
PRODUCT_LR_CORNER_LAT = 47.8615913
PRODUCT_LR_CORNER_LON = 87.4144676
PRODUCT_UL_CORNER_MAPX = 284400.000
PRODUCT_UL_CORNER_MAPY = 5524200.000
PRODUCT_UR_CORNER_MAPX = 531000.000
PRODUCT_UR_CORNER_MAPY = 5524200.000
PRODUCT_LL_CORNER_MAPX = 284400.000
PRODUCT_LL_CORNER_MAPY = 5301000.000
PRODUCT_LR_CORNER_MAPX = 531000.000
PRODUCT_LR_CORNER_MAPY = 5301000.000
PRODUCT_SAMPLES_REF = 8221
PRODUCT_LINES_REF = 7441
PRODUCT_SAMPLES_THM = 4111
PRODUCT_LINES_THM = 3721
BAND1_FILE_NAME = "L5145026_02620100615_B10.TIF"
BAND2_FILE_NAME = "L5145026_02620100615_B20.TIF"
BAND3_FILE_NAME = "L5145026_02620100615_B30.TIF"
BAND4_FILE_NAME = "L5145026_02620100615_B40.TIF"
BAND5_FILE_NAME = "L5145026_02620100615_B50.TIF"
BAND6_FILE_NAME = "L5145026_02620100615_B60.TIF"
BAND7_FILE_NAME = "L5145026_02620100615_B70.TIF"
GCP_FILE_NAME = "L5145026_02620100615_GCP.txt"
METADATA_L1_FILE_NAME = "L5145026_02620100615_MTL.txt"
CPF_FILE_NAME = "L5CPF20100401_20100630_09"
END_GROUP = PRODUCT_METADATA
GROUP = MIN_MAX_RADIANCE
LMAX_BAND1 = 193.000
LMIN_BAND1 = -1.520
LMAX_BAND2 = 365.000
LMIN_BAND2 = -2.840
LMAX_BAND3 = 264.000
LMIN_BAND3 = -1.170
LMAX_BAND4 = 221.000
LMIN_BAND4 = -1.510
LMAX_BAND5 = 30.200
LMIN_BAND5 = -0.370
LMAX_BAND6 = 15.303
LMIN_BAND6 = 1.238
LMAX_BAND7 = 16.500
LMIN_BAND7 = -0.150
END_GROUP = MIN_MAX_RADIANCE
GROUP = MIN_MAX_PIXEL_VALUE
QCALMAX_BAND1 = 255.0
QCALMIN_BAND1 = 1.0
QCALMAX_BAND2 = 255.0
QCALMIN_BAND2 = 1.0
QCALMAX_BAND3 = 255.0
QCALMIN_BAND3 = 1.0
QCALMAX_BAND4 = 255.0
QCALMIN_BAND4 = 1.0
QCALMAX_BAND5 = 255.0
QCALMIN_BAND5 = 1.0
QCALMAX_BAND6 = 255.0
QCALMIN_BAND6 = 1.0
QCALMAX_BAND7 = 255.0
QCALMIN_BAND7 = 1.0
END_GROUP = MIN_MAX_PIXEL_VALUE
GROUP = PRODUCT_PARAMETERS
CORRECTION_METHOD_GAIN_BAND1 = "CPF"
CORRECTION_METHOD_GAIN_BAND2 = "CPF"
CORRECTION_METHOD_GAIN_BAND3 = "CPF"
CORRECTION_METHOD_GAIN_BAND4 = "CPF"
CORRECTION_METHOD_GAIN_BAND5 = "CPF"
CORRECTION_METHOD_GAIN_BAND6 = "IC"
CORRECTION_METHOD_GAIN_BAND7 = "CPF"
CORRECTION_METHOD_BIAS = "IC"
SUN_AZIMUTH = 141.2669762
SUN_ELEVATION = 59.9909680
OUTPUT_FORMAT = "GEOTIFF"
END_GROUP = PRODUCT_PARAMETERS
GROUP = CORRECTIONS_APPLIED
STRIPING_BAND1 = "NONE"
STRIPING_BAND2 = "NONE"
STRIPING_BAND3 = "NONE"
STRIPING_BAND4 = "NONE"
STRIPING_BAND5 = "NONE"
STRIPING_BAND6 = "NONE"
STRIPING_BAND7 = "NONE"
BANDING = "N"
COHERENT_NOISE = "N"
MEMORY_EFFECT = "Y"
SCAN_CORRELATED_SHIFT = "Y"
INOPERABLE_DETECTORS = "N"
DROPPED_LINES = "N"
END_GROUP = CORRECTIONS_APPLIED
GROUP = PROJECTION_PARAMETERS
REFERENCE_DATUM = "WGS84"
REFERENCE_ELLIPSOID = "WGS84"
GRID_CELL_SIZE_THM = 60.000
GRID_CELL_SIZE_REF = 30.000
ORIENTATION = "NUP"
RESAMPLING_OPTION = "CC"
MAP_PROJECTION = "UTM"
END_GROUP = PROJECTION_PARAMETERS
GROUP = UTM_PARAMETERS
ZONE_NUMBER = 45
END_GROUP = UTM_PARAMETERS
END_GROUP = L1_METADATA_FILE
END
You might first find the contour with the biggest area. Then try some algorithm to find the points you want. It seems that the satellite picture in the image is not a perfect rectangle, so you can't fit a rectangle on it using OpenCV's built-in methods.
You should try something like that:
import cv2
import numpy as np
img = cv2.imread('z_edited.jpg')
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(imgray, (11, 11), 0)
ret, thresh = cv2.threshold(blurred, 27, 255, 0)
cnts, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
max_area = 0
max_area_index = 0
for i, cnt in enumerate(cnts):
area = cv2.contourArea(cnt)
if area > max_area:
max_area = area
max_area_index = i
x_min = np.min(cnts[max_area_index][:, 0, 0])
x_max = np.max(cnts[max_area_index][:, 0, 0])
y_min = np.min(cnts[max_area_index][:, 0, 1])
y_max = np.max(cnts[max_area_index][:, 0, 1])
(x_left, y_left) = (x_min, cnts[max_area_index][np.max(np.where(cnts[max_area_index][:, 0, 0] == x_min)), 0, 1])
(x_right, y_right) = (x_max, cnts[max_area_index][np.max(np.where(cnts[max_area_index][:, 0, 0] == x_max)), 0, 1])
(x_down, y_down) = (cnts[max_area_index][np.max(np.where(cnts[max_area_index][:, 0, 1] == y_max)), 0, 0], y_max)
(x_top, y_top) = (cnts[max_area_index][np.max(np.where(cnts[max_area_index][:, 0, 1] == y_min)), 0, 0], y_min)
cv2.circle(img, (x_left, y_left), 10, (0, 0, 255), thickness=8)
cv2.circle(img, (x_right, y_right), 10, (0, 0, 255), thickness=8)
cv2.circle(img, (x_down, y_down), 10, (0, 0, 255), thickness=8)
cv2.circle(img, (x_top, y_top), 10, (0, 0, 255), thickness=8)
# cv2.drawContours(img, cnts, max_area_index, (0, 255, 0), 2)
cv2.namedWindow('s', cv2.WINDOW_NORMAL)
cv2.imshow('s', img)
cv2.waitKey(0)
And the result looks like:
Using this code you can find the coordinates of the corners of the satellite picture inside the image(red points).
Also need to say I have assumed that your satellite picture background is completely black(the image you have uploaded, has a thin gray strip around the whole image).
I want to display a highcharter stacked bar chart where the row labels are trimmed that the first five characters are not shown. However, in the tooltip the full category names should be shown.
In the example above as categories at the xAxis I would like to have only "2012", "2013",.., whereas in the tooltip the whole category names should be displayed.
Here is my code
bs.table = data.frame(
Closing.Date = c("Line 2012", "Year 2013", "Year 2014", "Year 2015", "Year 2016"),
Non.Current.Assets = c(40.4, 30.3, 20.4, 34.5, 20),
Current.Assets = c(3.2, 3.3, 2.4, 3.5, 2)
)
hc <- highchart() %>%
hc_chart(type = "bar") %>%
hc_plotOptions(series = list(stacking = "normal")) %>%
hc_xAxis(categories = bs.table$Closing.Date,
lineColor = 'transparent',
tickWidth = 0,
labels = list(enable = TRUE,
align = 'left',
x = 5,
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_add_series(name ="Non Current Assets",
data = bs.table$Current.Assets,
stack = "Assets",
dataLabels = list(enabled = TRUE,
inside = TRUE,
align = "right",
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_add_series(name = "Current Assets",
data = bs.table$Non.Current.Assets,
stack = "Assets",
dataLabels = list(enabled = TRUE, inside = FALSE, align = "right",
style = list(fontSize = '1em',color = '#fff')) ) %>%
hc_legend(enabled = FALSE) %>%
hc_tooltip(shared = TRUE,
headerFormat = '<b>Statement {point.x}</b><br>',
pointFormat = '<b>{series.name}:</b> {point.y} <br>',
footerFormat = '<b>Total: {point.total} </b>')
Many thanks in advance!
Could you not just change the column name before you create the chart?
# function to get year
substrRight <- function(x, n){
substr(x, nchar(x)-n+1, nchar(x))
}
# create year column
bs.table$year = substrRight(as.character(bs.table$Closing.Date), 4)
# alter x axis to use this column
hc <- highchart() %>%
hc_chart(type = "bar") %>%
hc_plotOptions(series = list(stacking = "normal")) %>%
hc_xAxis(categories = bs.table$year,
lineColor = 'transparent',
tickWidth = 0,
labels = list(enable = TRUE,
align = 'left',
x = 5,
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_add_series(name ="Non Current Assets",
data = bs.table$Current.Assets,
stack = "Assets",
dataLabels = list(enabled = TRUE,
inside = TRUE,
align = "right",
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_add_series(name = "Current Assets",
data = bs.table$Non.Current.Assets,
stack = "Assets",
dataLabels = list(enabled = TRUE, inside = FALSE, align = "right",
style = list(fontSize = '1em',color = '#fff')) ) %>%
hc_legend(enabled = FALSE) %>%
hc_tooltip(shared = TRUE,
headerFormat = '<b>Statement {point.x}</b><br>',
pointFormat = '<b>{series.name}:</b> {point.y} <br>',
footerFormat = '<b>Total: {point.total} </b>')
Edit
This is a sort of workaround that would nearly give you what you want:
highchart() %>%
hc_chart(type = "bar") %>%
hc_xAxis(categories = bs.table$year,
lineColor = 'transparent',
tickWidth = 0,
labels = list(enable = TRUE,
align = 'left',
x = 5,
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_plotOptions(series = list(stacking = "normal")) %>%
hc_add_series(name = "Current Assets", bs.table, "column", hcaes(x = year, y = Current.Assets, stuff = Closing.Date),
tooltip = list(pointFormat = "<b>{point.stuff}</b><br> <b>{series.name}:</b> {point.y} <br>"),
dataLabels = list(enabled = TRUE,
inside = TRUE,
align = "right",
style = list(fontSize = '1em',color = '#fff'))) %>%
hc_add_series(name ="Non Current Assets", bs.table, "column", hcaes(x = year, y = Non.Current.Assets),
tooltip = list(pointFormat = "<b>{point.stuff}</b><br>"),
dataLabels = list(enabled = TRUE, inside = FALSE, align = "right",
style = list(fontSize = '1em',color = '#fff')) ) %>%
hc_legend(enabled = FALSE) %>%
hc_tooltip(shared = TRUE,
headerFormat = '<b>Statement </b>',
footerFormat = '<b>Total: {point.total} </b>')
I used a code to scrape two pages from tripadvisor, and it worked very well. But now, it shows me two differents errors :
with open("iletaitunsquare1.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
w = csv.writer(csv_file, delimiter = ";", quoting=csv.QUOTE_MINIMAL)
w.writerow(["inf_rest_name", "rest_eclf", "name_client", "date_rev_cli", "opinion_cl"])
with requests. Session() as s:
for offset in range (270,1230,10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d6575305-Reviews-or{offset}-Il_Etait_Un_Square-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
reviews = soup.select('.reviewSelector')
ids = [review.get('data.reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'Referer': r.url}
)
soup = bs(r.content, 'lxml')
if not offset:
inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("\n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}", f"{titre_rev_cl}", f"{opinion_cl}"]
w.writerow(row)
Error on excecution:
"data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'}
TypeError: sequence item 0: expected str instance, NoneType found"
and after I decided to change just values in line 6(pages of site) and 7 (Url):
with open("boutary.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
w = csv.writer(csv_file, delimiter = ";", quoting=csv.QUOTE_MINIMAL)
w.writerow(["inf_rest_name", "rest_eclf", "name_client", "date_rev_cl", "titre_rev_cl", "opinion_cl"])
with requests.Session() as s:
for offset in range(40, 290, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d9783452-Reviews-or{offset}-Boutary-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'referer': r.url}
)
soup = bs(r.content, 'lxml')
if not offset:
inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("\n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]
w.writerow(row)
and it shows me
"row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}",
f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]
NameError: name 'inf_rest_name' is not defined"
These errors are strange because, before, I used the same code with other URL and it worked perfectly.
Can you tell me please what is happening? How can I run it proprerly? I will appreciate your help.
This is because in the original code, not posted here, it was relying on Truthy/Falsy value of offset 0 which in your prior question was the first offset.
For example, with:
for offset in range(0, 10, 10):
if not offset:
The first value 0 is a Falsy versus numbers > 0 (in this scenario) which will be seen as Truthy. If not True i.e. False i.e. if 0 offset then set the value of inf_rest_name. This ensures its value it only set on the first loop rather than each time. Its value doesn't change so no need to read again.
With the following all values are Truthies and so inf_rest_name never gets set.
for offset in range(40, 290, 10):
if not offset:
You could change to:
if offset == firstvalue:
e.g.
if offset == 40:
inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
See this for more info.
Those lines also need to work with first soup not later soup (as that is only reviews)
import requests
from bs4 import BeautifulSoup as bs
with requests.Session() as s:
for offset in range(40, 290, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d9783452-Reviews-or{offset}-Boutary-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
if offset == 40:
inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'referer': r.url}
)
soup = bs(r.content, 'lxml')
for review in soup.select('.reviewSelector'):
name_client = review.select_one('.info_text > div:first-child').text.strip()
date_rev_cl = review.select_one('.ratingDate')['title'].strip()
titre_rev_cl = review.select_one('.noQuotes').text.strip()
opinion_cl = review.select_one('.partial_entry').text.replace("\n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}" , f"{titre_rev_cl}", f"{opinion_cl}"]
For your first code block you are using an invalid attribute. It should be
ids = [review.get('data-reviewid') for review in reviews]
Note I have added an is None test to handle not found. This should be added to top version as well.
import requests
from bs4 import BeautifulSoup as bs
with requests. Session() as s:
for offset in range (270, 1230, 10):
url = f'https://www.tripadvisor.fr/Restaurant_Review-g187147-d6575305-Reviews-or{offset}-Il_Etait_Un_Square-Paris_Ile_de_France.html'
r = s.get(url)
soup = bs(r.content, 'lxml')
if offset == 270:
inf_rest_name = soup.select_one('.heading').text.replace("\n","").strip()
rest_eclf = soup.select_one('.header_links a').text.strip()
reviews = soup.select('.reviewSelector')
ids = [review.get('data-reviewid') for review in reviews]
r = s.post(
'https://www.tripadvisor.fr/OverlayWidgetAjax?Mode=EXPANDED_HOTEL_REVIEWS_RESP&metaReferer=',
data = {'reviews': ','.join(ids), 'contextChoice': 'DETAIL'},
headers = {'Referer': r.url}
)
soup = bs(r.content, 'lxml')
for review in soup.select('.reviewSelector'):
name_client= review.select_one('.info_text > div:first-child')
if name_client is None:
name_client = 'N/A'
else:
name_client = name_client.text.strip()
date_rev_cl = review.select_one('.ratingDate')
if date_rev_cl is None:
date_rev_cl = 'N/A'
else:
date_rev_cl = date_rev_cl['title'].strip()
titre_rev_cl = review.select_one('.noQuotes')
if titre_rev_cl is None:
titre_rev_cl = 'N/A'
else:
titre_rev_cl = titre_rev_cl.text.strip()
opinion_cl = review.select_one('.partial_entry')
if opinion_cl is None:
opinion_cl = 'N/A'
else:
opinion_cl = opinion_cl.text.replace("\n","").strip()
row = [f"{inf_rest_name}", f"{rest_eclf}", f"{name_client}", f"{date_rev_cl}", f"{titre_rev_cl}", f"{opinion_cl}"]
print(row)
How do I plot excel bar chart from Matlab?
I'm able to plot line chart.
e = actxserver('excel.application');
eWs = e.Workbooks;
eW = eWs.Add;
eS = eW.ActiveSheet;
e.Visible = 1;
x=(0:2:100)';y=sin(x);
eS.Range('A1:B50').Value = [x y];
eCO = eS.ChartObjects.Add(100, 30, 400, 250);
eC = eCO.Chart;
eC.SeriesCollection.NewSeries;
eC.SeriesCollection(1).Value = eS.Range('B1:B50');
eC.SeriesCollection(1).XValue = eS.Range('A1:A50');
eCO.Chart.ChartType = 1;
eCO.Chart.ChartType = 65;
eCO.Chart.HasTitle = true;
eCO.Chart.ChartTitle.Text = 'This is the title text'; % view it again
eW.Close;e.Quit;delete(e);
Ok, I figured it out just after posting it.
We can get this by changing
eCO.Chart.ChartType = 65; to eCO.Chart.ChartType = 57;
e = actxserver('excel.application');
eWs = e.Workbooks;
eW = eWs.Add;
eS = eW.ActiveSheet;
e.Visible = 1;
x=(0:2:100)';y=sin(x);
eS.Range('A1:B50').Value = [x y];
eCO = eS.ChartObjects.Add(100, 30, 400, 250);
eC = eCO.Chart;
eC.SeriesCollection.NewSeries;
eC.SeriesCollection(1).Value = eS.Range('B1:B50');
eC.SeriesCollection(1).XValue = eS.Range('A1:A50');
eCO.Chart.ChartType = 1;
eCO.Chart.ChartType = 57;
eCO.Chart.HasTitle = true;
eCO.Chart.ChartTitle.Text = 'This is the title text'; % view it again
eW.Close;e.Quit;delete(e);
For further reference to other charts check this http://it.toolbox.com/wiki/index.php/EXCEL_Chart_Type_Enumeration