I am getting delivery fee but for few restaurants, its slightly different than the one on website, say its $2.99 on website but spider is getting $3.09. I have checked latitude and longitude but its same in the url and spider. I have changed multiple values in headers and it changed the delivery fee for few restaurants but not for all. This is the sample url: https://www.foodpanda.sg/restaurants/new?lat=1.43035&lng=103.83684&vertical=restaurants and below is the code snippet:
api_url = 'https://disco.deliveryhero.io/listing/api/v1/pandora/vendors'
form_data = {
'country': 'sg',
'latitude': str(lat),
'longitude': str(lng),
'sort': '',
'language_id': str(self.language_id),
'vertical': 'restaurants',
'include': 'characteristics',
'dynamic_pricing': '0',
'configuration': 'Variant1',
'use_free_delivery_label': 'true',
'limit': '50',
'offset': '0'
}
headers = {
'X-FP-API-KEY': 'volo',
'dps-session-id': dps_session_id,
'x-disco-client-id': 'web',
}
return scrapy.FormRequest(
api_url,
method='GET',
headers=headers,
formdata=form_data,
meta={
'page': page,
'lat': lat,
'lng': lng,
'item_meta': item_meta,
'start_position': start_position
},
callback=self.parse_search
)
Any clue or insight is appreciated.
Do you have an idea of how I can round float numbers after multiplying?
I have the following SQL dump:
INSERT INTO
`honzavolfcz_product` (`product_id`, `feed_product_id`, `import_id`,
`import_active_product`, `model`, `sku`, `upc`, `ean`, `jan`, `isbn`, `mpn`,
`location`, `quantity`, `stock_status_id`, `product_status_id`, `image`,
`manufacturer_id`, `shipping`, `price`, `points`, `tax_class_id`,
`date_available`, `weight`, `weight_class_id`, `length`, `width`, `height`,
`length_class_id`, `subtract`, `minimum`, `sort_order`, `status`, `date_added`,
`date_modified`, `viewed`)
VALUES ('10', '0', '1',
'1', 'model', '', '', '', '', '', '',
'', '1', '1', '0', 'catalog/zbozi/bozi_laska_obal.jpg',
'0', '1', '**112.50**', '0', '1',
'2019-01-15', '0.00', '1', '0.00', '0.00', '0.00',
'1', '0', '1', '0', '1', '2019-02-15 16:16:29',
'2019-02-15 16:16:29', '293');
And I want to multiply the price value (112.50) by 1.21 (taxes) and the round-up or down. I wrote the following command which does the multiplication but I do not know how to round it:
awk '{a=substr($58,2,length($58)-3);gsub(a,a*1.21);print}' a > b
The result:
INSERT INTO
`honzavolfcz_product` (`product_id`, `feed_product_id`, `import_id`,
`import_active_product`, `model`, `sku`, `upc`, `ean`, `jan`, `isbn`, `mpn`,
`location`, `quantity`, `stock_status_id`, `product_status_id`, `image`,
`manufacturer_id`, `shipping`, `price`, `points`, `tax_class_id`,
`date_available`, `weight`, `weight_class_id`, `length`, `width`, `height`,
`length_class_id`, `subtract`, `minimum`, `sort_order`, `status`, `date_added`,
`date_modified`, `viewed`)
VALUES ('10', '0', '1',
'1', 'model', '', '', '', '', '', '',
'', '1', '1', '0', 'catalog/zbozi/bozi_laska_obal.jpg',
'0', '1', '**136.125**', '0', '1',
'2019-01-15', '0.00', '1', '0.00', '0.00', '0.00',
'1', '0', '1', '0', '1', '2019-02-15 16:16:29',
'2019-02-15 16:16:29', '293');
I would like to have there 136 instead of 136.125. Of course, 137 if it would be 136.555.
Thank you in advance.
This may be what you want:
$ awk '{a=substr($58,2); $58=sprintf("\047%d\047,",a*1.21)} 1' file
INSERT INTO honzavolfcz_product (product_id, feed_product_id, import_id, import_active_product, model, sku, upc, ean, jan, isbn, mpn, location, quantity, stock_status_id, product_status_id, image, manufacturer_id, shipping, price, points, tax_class_id, date_available, weight, weight_class_id, length, width, height, length_class_id, subtract, minimum, sort_order, status, date_added, date_modified, viewed) VALUES ('10', '0', '1', '1', 'model', '', '', '', '', '', '', '', '1', '1', '0', 'catalog/zbozi/bozi_laska_obal.jpg', '0', '1', '136', '0', '1', '2019-01-15', '0.00', '1', '0.00', '0.00', '0.00', '1', '0', '1', '0', '1', '2019-02-15 16:16:29', '2019-02-15 16:16:29', '293');
but the rounding probably won't go quite as you'd like by default. See https://www.gnu.org/software/gawk/manual/gawk.html#Round-Function and https://www.gnu.org/software/gawk/manual/gawk.html#Setting-the-rounding-mode for how to control it with GNU awk.
I can't convert this Json to csv. I have been trying with different solutions posted here using panda or other parser but non solved this.
This is a small extract of the big json
{'data': {'items': [{'category': 'cat',
'coupon_code': 'cupon 1',
'coupon_name': '$829.99/€705.79 ',
'coupon_url': 'link3',
'end_time': '2017-12-31 00:00:00',
'language': 'sp',
'start_time': '2017-12-19 00:00:00'},
{'category': 'LED ',
'coupon_code': 'code',
'coupon_name': 'text',
'coupon_url': 'link',
'end_time': '2018-01-31 00:00:00',
'language': 'sp',
'start_time': '2017-10-07 00:00:00'}],
'total_pages': 1,
'total_results': 137},
'error_no': 0,
'msg': '',
'request': 'GET api/ #2017-12-26 04:50:02'}
I'd like to get an output like this with the columns:
category, coupon_code, coupon_name, coupon_url, end_time, language, start_time
I'm running python 3.6 with no restrictions.
I have next Cassandra table structure:
CREATE TABLE ringostat.hits (
hitId uuid,
clientId VARCHAR,
session MAP<VARCHAR, TEXT>,
traffic MAP<VARCHAR, TEXT>,
PRIMARY KEY (hitId, clientId)
);
INSERT INTO ringostat.hits (hitId, clientId, session, traffic)
VALUES('550e8400-e29b-41d4-a716-446655440000'. 'clientId', {'id': '1', 'number': '1', 'startTime': '1460023732', 'endTime': '1460023762'}, {'referralPath': '/example_path_for_example', 'campaign': '(not set)', 'source': 'www.google.com', 'medium': 'referal', 'keyword': '(not set)', 'adContent': '(not set)', 'campaignId': '', 'gclid': '', 'yclid': ''});
INSERT INTO ringostat.hits (hitId, clientId, session, traffic)
VALUES('650e8400-e29b-41d4-a716-446655440000'. 'clientId', {'id': '1', 'number': '1', 'startTime': '1460023732', 'endTime': '1460023762'}, {'referralPath': '/example_path_for_example', 'campaign': '(not set)', 'source': 'www.google.com', 'medium': 'cpc', 'keyword': '(not set)', 'adContent': '(not set)', 'campaignId': '', 'gclid': '', 'yclid': ''});
INSERT INTO ringostat.hits (hitId, clientId, session, traffic)
VALUES('750e8400-e29b-41d4-a716-446655440000'. 'clientId', {'id': '1', 'number': '1', 'startTime': '1460023732', 'endTime': '1460023762'}, {'referralPath': '/example_path_for_example', 'campaign': '(not set)', 'source': 'www.google.com', 'medium': 'referal', 'keyword': '(not set)', 'adContent': '(not set)', 'campaignId': '', 'gclid': '', 'yclid': ''});
I want to select all rows where source='www.google.com' AND medium='referal'.
SELECT * FROM hits WHERE traffic['source'] = 'www.google.com' AND traffic['medium'] = 'referal' ALLOW FILTERING;
Without add ALLOW FILTERING I got error: No supported secondary index found for the non primary key columns restrictions.
That's why I see two options:
Create index on traffic column.
Create materialized view.
Create another table and set INDEX for traffic column.
Which is the best option ? Also, I have many fields with MAP type on which I will need to filter. What issues can be if on every field I will add INDEX ?
Thank You.
From When to use an index.
Do not use an index in these situations:
On high-cardinality columns because you then query a huge volume of records for a small number of results. [...] Conversely, creating an index on an extremely low-cardinality column, such as a boolean column, does not make sense.
In tables that use a counter column
On a frequently updated or deleted column.
To look for a row in a large partition unless narrowly queried.
If your planned usage meets one or more of these criteria, it is probably better to use a materialized view.