Python fbprophet - export values from plot_components() for yearly - python-3.x

Any ideas on how to export the yearly seasonal trend using fbprophet library?
The plot_components() function plots the trend, yearly, and weekly.
I want to obtain the values for yearly only.


Example data:
import pandas as pd
from fbprophet import Prophet
import matplotlib.pyplot as plt
df = pd.DataFrame.from_dict({'ds': ['1949-01', '1949-02', '1949-03', '1949-04', '1949-05', '1949-06',
'1949-07', '1949-08', '1949-09', '1949-10', '1949-11', '1949-12',
'1950-01', '1950-02', '1950-03', '1950-04', '1950-05', '1950-06',
'1950-07', '1950-08', '1950-09', '1950-10', '1950-11', '1950-12',
'1951-01', '1951-02', '1951-03', '1951-04', '1951-05', '1951-06',
'1951-07', '1951-08', '1951-09', '1951-10', '1951-11', '1951-12',
'1952-01', '1952-02', '1952-03', '1952-04', '1952-05', '1952-06',
'1952-07', '1952-08', '1952-09', '1952-10', '1952-11', '1952-12',
'1953-01', '1953-02', '1953-03', '1953-04', '1953-05', '1953-06',
'1953-07', '1953-08', '1953-09', '1953-10', '1953-11',
'1953-12',
'1954-01', '1954-02', '1954-03', '1954-04', '1954-05', '1954-06',
'1954-07', '1954-08', '1954-09', '1954-10', '1954-11', '1954-12',
'1955-01', '1955-02', '1955-03', '1955-04', '1955-05', '1955-06',
'1955-07', '1955-08', '1955-09', '1955-10', '1955-11', '1955-12',
'1956-01', '1956-02', '1956-03', '1956-04', '1956-05', '1956-06',
'1956-07', '1956-08', '1956-09', '1956-10', '1956-11', '1956-12',
'1957-01', '1957-02', '1957-03', '1957-04', '1957-05', '1957-06',
'1957-07', '1957-08', '1957-09', '1957-10', '1957-11', '1957-12',
'1958-01', '1958-02', '1958-03', '1958-04', '1958-05', '1958-06',
'1958-07', '1958-08', '1958-09', '1958-10', '1958-11', '1958-12',
'1959-01', '1959-02', '1959-03', '1959-04', '1959-05', '1959-06',
'1959-07', '1959-08', '1959-09', '1959-10', '1959-11', '1959-12',
'1960-01', '1960-02', '1960-03', '1960-04', '1960-05', '1960-06',
'1960-07', '1960-08', '1960-09', '1960-10', '1960-11', '1960-12'],
'y': [112, 118, 132, 129, 121, 135, 148, 148, 136, 119, 104, 118, 115, 126,
141, 135, 125, 149, 170, 170, 158, 133, 114, 140, 145, 150, 178, 163,
172, 178, 199, 199, 184, 162, 146, 166, 171, 180, 193, 181, 183, 218,
230, 242, 209, 191, 172, 194, 196, 196, 236, 235, 229, 243, 264, 272,
237, 211, 180, 201, 204, 188, 235, 227, 234, 264, 302, 293, 259, 229,
203, 229, 242, 233, 267, 269, 270, 315, 364, 347, 312, 274, 237, 278,
284, 277, 317, 313, 318, 374, 413, 405, 355, 306, 271, 306, 315, 301,
356, 348, 355, 422, 465, 467, 404, 347, 305, 336, 340, 318, 362, 348,
363, 435, 491, 505, 404, 359, 310, 337, 360, 342, 406, 396, 420, 472,
548, 559, 463, 407, 362, 405, 417, 391, 419, 461, 472, 535, 622, 606,
508, 461, 390, 432]})
df['ds'] = pd.to_datetime(df['ds'])
fbprophet plot_components():
model = Prophet()
model.fit(df)
future = model.make_future_dataframe(12, 'm')
fc = model.predict(future)
model.plot_components(fc)
plt.show()
I understand your question to mean "How do we get the values used in the "yearly" plot above?
days = (pd.date_range(start='2017-01-01', periods=365) + pd.Timedelta(days=0))
df_y = model.seasonality_plot_df(days)
seas = model.predict_seasonal_components(df_y)
fig,ax = plt.subplots(2, 1, figsize=(8,6))
ax[0].plot(fc['ds'].dt.to_pydatetime(), fc['trend'])
ax[0].grid(alpha=0.5)
ax[0].set_xlabel('ds')
ax[1].set_ylabel('trend')
ax[1].plot(df_y['ds'].dt.to_pydatetime(), seas['yearly'], ls='-', c='#0072B2')
ax[1].set_xlabel('Day of year')
ax[1].set_ylabel('yearly')
ax[1].grid(alpha=0.5)
plt.show()
Plot results:
How to obtain the values for yearly only?
The X values are an arbitrary yearly time period (basically). The y values use functions seasonality_plot_df() and predict_seasonal_components() to predict daily seasonality for a year time-span. You retrieve these values by looking in seas['yearly'].


There is a simple solution in the current version of the library. You can use from the predicted model fc. What you want for the value of yearly can be found with fc['yearly'] without using the functions in the above solution.
Moreover, if you want all the other components like trend, you can use fc['trend'].

Related

Estimating Region covered by all sides

I need some help in estimating somekind of of morphological operations(preferably using skimage or scipy image or python in general) that can help me isolate the regions that are enclosed by all sides(see images.) The problem is the outer shell can be of varied shape and size. The enclosed regions are also of varied shapes and sizes.
Examples of the image where the enclosed(by the white regions) black region needs to isolated.
By isolation I mean to get all the pixel coordinates in the enclosed black regions.
Example 1
Example 2
Also another caveat is that the images like this should not be filled up.

Use scipy.ndimage.binary_fill_holes
from scipy import ndimage as ndi
filled = ndi.binary_fill_holes(image)
holes = filled - image # or np.logical_xor(filled, image)

I prefer #Juan's solution, but just for fun, here's a slightly different approach. Floodfill with white starting from the top-left corner and your fully-enclosed pixels will be all that is left over.
#!/usr/bin/env python3
import numpy as np
from skimage.segmentation import flood_fill
import cv2
im = cv2.imread('scan.png', cv2.IMREAD_GRAYSCALE)
# Fill with white from top-left
filled = flood_fill(im, (0, 0), 255)
# Save result
cv2.imwrite('DEBUG-result.png', filled)
# Get list of black pixels
blk = np.where(filled==0)
print(f'Number of fully enclosed black pixels: {len(blk[0])}')
print(blk)
So that you can see the extent of the image, I have artificially added a red border:
Here is the output:
Number of fully enclosed black pixels: 957
(array([364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364,
364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364,
364, 364, 364, 364, 364, 364, 364, 365, 365, 365, 365, 365, 365,
365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365,
365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365,
365, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366,
366, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366, 366,
366, 366, 366, 366, 366, 366, 366, 366, 367, 367, 367, 367, 367,
367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367,
367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367, 367,
367, 367, 367, 367, 367, 368, 368, 368, 368, 368, 368, 368, 368,
368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368,
368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368, 368,
368, 368, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369,
369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369,
369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 369, 370,
370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370,
370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370, 370,
370, 370, 370, 370, 370, 370, 370, 370, 370, 371, 371, 371, 371,
371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371,
371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371, 371,
371, 371, 371, 371, 371, 371, 372, 372, 372, 372, 372, 372, 372,
372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372,
372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372, 372,
372, 372, 372, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373,
373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373,
373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 374, 374, 374,
374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374,
374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374, 374,
374, 374, 374, 374, 375, 375, 375, 375, 375, 375, 375, 375, 375,
375, 375, 375, 375, 375, 375, 375, 375, 375, 375, 375, 375, 375,
375, 375, 375, 375, 375, 375, 375, 375, 375, 375, 375, 376, 376,
376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376,
376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376, 376,
376, 376, 376, 376, 376, 377, 377, 377, 377, 377, 377, 377, 377,
377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 377,
377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, 378,
378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378,
378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378, 378,
378, 378, 378, 378, 378, 378, 379, 379, 379, 379, 379, 379, 379,
379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379,
379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379, 379,
380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380,
380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380, 380,
380, 380, 380, 380, 380, 380, 380, 381, 381, 381, 381, 381, 381,
381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381,
381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381, 381,
381, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382,
382, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382, 382,
382, 382, 382, 382, 382, 382, 382, 382, 383, 383, 383, 383, 383,
383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383,
383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383, 383,
383, 383, 384, 384, 384, 384, 384, 384, 384, 384, 384, 384, 384,
384, 384, 384, 384, 384, 384, 384, 384, 384, 384, 384, 384, 384,
384, 384, 384, 384, 384, 384, 384, 384, 384, 385, 385, 385, 385,
385, 385, 385, 385, 385, 385, 385, 385, 385, 385, 385, 385, 385,
385, 385, 385, 385, 385, 385, 385, 386, 386, 386, 386, 386, 386,
386, 386, 386, 386, 386, 386, 386, 386, 386, 386, 386, 386, 386,
386, 386, 386, 386, 386, 387, 387, 387, 387, 387, 387, 387, 387,
387, 387, 387, 387, 387, 387, 387, 387, 387, 387, 387, 387, 387,
387, 387, 387, 388, 388, 388, 388, 388, 388, 388, 388, 388, 388,
388, 388, 388, 388, 388, 388, 388, 388, 388, 388, 388, 389, 389,
389, 389, 389, 389, 389, 389, 389, 389, 389, 389, 389, 389, 389,
389, 389, 389, 389, 389, 389, 390, 390, 390, 390, 390, 390, 390,
390, 390, 390, 390, 390, 390, 390, 390, 390, 390, 390, 390, 390,
390, 391, 391, 391, 391, 391, 391, 391, 391, 391, 391, 391, 391,
391, 391, 391, 392, 392, 392, 392, 392, 392, 392, 392, 392, 392,
392, 392, 392, 392, 392, 393, 393, 393, 393, 393, 393, 393, 393,
393, 393, 393, 393, 393, 393, 393, 394, 394, 394, 394, 394, 394,
394, 394, 394, 394, 394, 394, 394, 394, 394, 395, 395, 395, 395,
395, 395, 395, 395, 395, 395, 395, 395, 395, 395, 395, 396, 396,
396, 396, 396, 396, 396, 396, 396, 397, 397, 397, 397, 397, 397,
397, 397, 397, 398, 398, 398, 398, 398, 398, 398, 398, 398, 399,
399, 399, 400, 400, 400, 401, 401, 401]), array([244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256,
257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269,
270, 271, 272, 273, 274, 275, 276, 244, 245, 246, 247, 248, 249,
250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275,
276, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,
256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268,
269, 270, 271, 272, 273, 274, 275, 276, 241, 242, 243, 244, 245,
246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271,
272, 273, 274, 275, 276, 241, 242, 243, 244, 245, 246, 247, 248,
249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261,
262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274,
275, 276, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264,
265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 241,
242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254,
255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267,
268, 269, 270, 271, 272, 273, 274, 275, 276, 241, 242, 243, 244,
245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257,
258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270,
271, 272, 273, 274, 275, 276, 241, 242, 243, 244, 245, 246, 247,
248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260,
261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273,
274, 275, 276, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250,
251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263,
264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 241, 242, 243,
244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256,
257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269,
270, 271, 272, 273, 241, 242, 243, 244, 245, 246, 247, 248, 249,
250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 241, 242,
243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,
256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268,
269, 270, 271, 272, 273, 241, 242, 243, 244, 245, 246, 247, 248,
249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261,
262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 241,
242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254,
255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267,
268, 269, 270, 271, 272, 273, 241, 242, 243, 244, 245, 246, 247,
248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260,
261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273,
241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253,
254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266,
267, 268, 269, 270, 271, 272, 273, 241, 242, 243, 244, 245, 246,
247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259,
260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
273, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252,
253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
266, 267, 268, 269, 270, 271, 272, 273, 241, 242, 243, 244, 245,
246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271,
272, 273, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264,
265, 266, 267, 268, 269, 270, 271, 272, 273, 253, 254, 255, 256,
257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269,
270, 271, 272, 273, 274, 275, 276, 253, 254, 255, 256, 257, 258,
259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271,
272, 273, 274, 275, 276, 253, 254, 255, 256, 257, 258, 259, 260,
261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273,
274, 275, 276, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 256, 257,
258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270,
271, 272, 273, 274, 275, 276, 256, 257, 258, 259, 260, 261, 262,
263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275,
276, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270,
271, 272, 273, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268,
269, 270, 271, 272, 273, 259, 260, 261, 262, 263, 264, 265, 266,
267, 268, 269, 270, 271, 272, 273, 259, 260, 261, 262, 263, 264,
265, 266, 267, 268, 269, 270, 271, 272, 273, 259, 260, 261, 262,
263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 262, 263,
264, 265, 266, 267, 268, 269, 270, 262, 263, 264, 265, 266, 267,
268, 269, 270, 262, 263, 264, 265, 266, 267, 268, 269, 270, 265,
266, 267, 265, 266, 267, 265, 266, 267]))
Hopefully you can see from the first element of each array that the first black pixel is at 364,244
Note that, if there is the possibility that the scan would touch the edge of the image, the flood-fill will fail to "flow" all the way around the edges. In this case, you can simply add a 1-pixel wide black border all the way around the edge for the floodfill to flow around... and remove it afterwards.

ortools vrp does not give me any solution

I want solve a vehicle routing problem with ORTools, both distance and duration matrix will be used.
but the problem is when I change the matrix , it wouldn't give me any solutions anymore!
there are 2 groups of matrixes. with the commented matrixes there is solution, but with the other group, there is not. do you have any idea why this is happening:
from __future__ import print_function
from ortools.constraint_solver import routing_enums_pb2
from ortools.constraint_solver import pywrapcp
def create_data_model():
"""Stores the data for the problem."""
data = {}
#data['distance_matrix']=[[0, 329, 146, 157, 318, 528, 457, 242, 491, 335, 471, 456, 391, 128, 461, 555, 460], [329, 0, 399, 384, 544, 493, 339, 378, 108, 243, 125, 394, 136, 561, 505, 315, 447], [146, 399, 0, 262, 471, 316, 297, 227, 548, 377, 267, 430, 383, 154, 234, 188, 400], [157, 384, 262, 0, 440, 271, 383, 525, 223, 367, 511, 354, 112, 539, 159, 152, 373], [318, 544, 471, 440, 0, 423, 112, 381, 346, 512, 161, 239, 581, 291, 284, 145, 143], [528, 493, 316, 271, 423, 0, 380, 196, 409, 212, 199, 277, 387, 515, 391, 261, 318], [457, 339, 297, 383, 112, 380, 0, 379, 298, 267, 482, 247, 462, 256, 296, 533, 200], [242, 378, 227, 525, 381, 196, 379, 0, 156, 230, 551, 555, 338, 372, 403, 358, 506], [491, 108, 548, 223, 346, 409, 298, 156, 0, 140, 532, 405, 531, 129, 220, 482, 222], [335, 243, 377, 367, 512, 212, 267, 230, 140, 0, 418, 440, 526, 255, 455, 296, 430], [471, 125, 267, 511, 161, 199, 482, 551, 532, 418, 0, 439, 285, 181, 254, 208, 304], [456, 394, 430, 354, 239, 277, 247, 555, 405, 440, 439, 0, 397, 229, 121, 385, 147], [391, 136, 383, 112, 581, 387, 462, 338, 531, 526, 285, 397, 0, 544, 205, 197, 226], [128, 561, 154, 539, 291, 515, 256, 372, 129, 255, 181, 229, 544, 0, 150, 204, 516], [461, 505, 234, 159, 284, 391, 296, 403, 220, 455, 254, 121, 205, 150, 0, 192, 544], [555, 315, 188, 152, 145, 261, 533, 358, 482, 296, 208, 385, 197, 204, 192, 0, 138], [460, 447, 400, 373, 143, 318, 200, 506, 222, 430, 304, 147, 226, 516, 544, 138, 0]]
data['distance_matrix']=[[0, 228, 299, 301, 235, 208, 405, 447, 144, 579], [228, 0, 343, 288, 357, 426, 530, 510, 122, 490], [299, 343, 0, 236, 228, 523, 274, 377, 397, 530], [301, 288, 236, 0, 594, 523, 289, 397, 154, 380], [235, 357, 228, 594, 0, 558, 370, 444, 173, 558], [208, 426, 523, 523, 558, 0, 219, 278, 504, 507], [405, 530, 274, 289, 370, 219, 0, 195, 283, 257], [447, 510, 377, 397, 444, 278, 195, 0, 407, 417], [144, 122, 397, 154, 173, 504, 283, 407, 0, 273], [579, 490, 530, 380, 558, 507, 257, 417, 273, 0]]
data['time_matrix']=[[0, 205, 519, 308, 428, 574, 399, 138, 573, 541], [205, 0, 447, 578, 296, 536, 135, 345, 198, 315], [519, 447, 0, 209, 438, 174, 231, 382, 104, 522], [308, 578, 209, 0, 235, 264, 492, 305, 134, 538], [428, 296, 438, 235, 0, 600, 177, 435, 204, 556], [574, 536, 174, 264, 600, 0, 476, 119, 183, 476], [399, 135, 231, 492, 177, 476, 0, 497, 208, 167], [138, 345, 382, 305, 435, 119, 497, 0, 344, 454], [573, 198, 104, 134, 204, 183, 208, 344, 0, 422], [541, 315, 522, 538, 556, 476, 167, 454, 422, 0]]
data['cost_matrix']=[[0, 160, 135, 433, 581, 453, 336, 329, 343, 237], [160, 0, 313, 596, 576, 458, 264, 380, 348, 354], [135, 313, 0, 591, 391, 211, 561, 236, 304, 414], [433, 596, 591, 0, 539, 253, 427, 300, 214, 118], [581, 576, 391, 539, 0, 243, 521, 499, 560, 255], [453, 458, 211, 253, 243, 0, 571, 216, 121, 314], [336, 264, 561, 427, 521, 571, 0, 425, 271, 165], [329, 380, 236, 300, 499, 216, 425, 0, 425, 549], [343, 348, 304, 214, 560, 121, 271, 425, 0, 176], [237, 354, 414, 118, 255, 314, 165, 549, 176, 0]]
data['num_vehicles'] = 4
data['depot'] = 0
return data
def print_solution(data, manager, routing, assignment):
"""Prints assignment on console."""
total_cost ,total_distance,total_time= 0,0,0
print('Objective: {}'.format(assignment.ObjectiveValue()))
distance_dimension=routing.GetDimensionOrDie('Distance')
time_dimension=routing.GetDimensionOrDie('Time')
for vehicle_id in range(data['num_vehicles']):
index = routing.Start(vehicle_id)
plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
route_cost = 0
route_distance = 0
route_time = 0
while not routing.IsEnd(index):
plan_output += ' {} -> '.format(manager.IndexToNode(index))
distance_var=distance_dimension.CumulVar(index)
time_var=time_dimension.CumulVar(index)
previous_index = index
index = assignment.Value(routing.NextVar(index))
route_cost += routing.GetArcCostForVehicle(previous_index, index, vehicle_id)
route_distance+=assignment.Value(distance_var)
route_time+=assignment.Value(time_var)
plan_output += '{}\n'.format(manager.IndexToNode(index))
plan_output += 'Cost of the route: {0}\nDistance of the route: {1}m\nTime of route: {2}\n'.format(
route_cost,
route_distance,
route_time)
print(plan_output)
total_cost += route_cost
total_time+=route_time
total_distance+=route_distance
print('Total Cost of all routes: {}\nTotal Distance of all routes: {}\nTotal Time of all routes: {}\n'.format(total_cost,total_distance,total_time))
def get_routes(manager, routing, solution, num_routes):
"""Get vehicle routes from a solution and store them in an array."""
# Get vehicle routes and store them in a two dimensional array whose
# i,j entry is the jth location visited by vehicle i along its route.
routes = []
for route_nbr in range(num_routes):
index = routing.Start(route_nbr)
route = [manager.IndexToNode(index)]
while not routing.IsEnd(index):
index = solution.Value(routing.NextVar(index))
route.append(manager.IndexToNode(index))
routes.append(route)
return routes
def main():
# Instantiate the data problem.
data = create_data_model()
# Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(len(data['cost_matrix']), data['num_vehicles'], data['depot'])
# Create Routing Model.
routing = pywrapcp.RoutingModel(manager)
# Create and register a transit callback.
def cost_callback(from_index, to_index):
"""Returns the distance between the two nodes."""
# Convert from routing variable Index to distance matrix NodeIndex.
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return data['cost_matrix'][from_node][to_node]
transit_callback_index = routing.RegisterTransitCallback(cost_callback)
routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)
# Add Cost constraint.
routing.AddDimension(
transit_callback_index,
0, # no slack
3000, # vehicle maximum travel distance
False, # start cumul to zero
'Cost')
cost_dimension = routing.GetDimensionOrDie('Cost')
cost_dimension.SetGlobalSpanCostCoefficient(1000)
#Add Distance constraint.
def distance_callback(from_index,to_index):
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return data['distance_matrix'][from_node][to_node]
distance_callback_index=routing.RegisterTransitCallback(distance_callback)
routing.AddDimension(
distance_callback_index,
0,
3000,
False,
'Distance')
distance_dimension=routing.GetDimensionOrDie('Distance')
#Add Time constraint.
def time_callback(from_index,to_index):
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return data['time_matrix'][from_node][to_node]
time_callback_index=routing.RegisterTransitCallback(time_callback)
routing.AddDimension(
time_callback_index,
0,
300,
False,
'Time')
time_dimension=routing.GetDimensionOrDie('Time')
# Setting first solution heuristic.
search_parameters = pywrapcp.DefaultRoutingSearchParameters()
search_parameters.first_solution_strategy = (routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC)
search_parameters.solution_limit = 100
search_parameters.time_limit.seconds = 3
# Solve the problem.
assignment = routing.SolveWithParameters(search_parameters)
# Print solution on console.
if assignment:
print_solution(data, manager, routing, assignment)
routes = get_routes(manager, routing, assignment, data['num_vehicles'])
# Display the routes.
for i, route in enumerate(routes):
print('Route', i, route)
if __name__ == '__main__':
main()

None could mean that it did not find a solution. Most likely your limits are too low
by increasing the limits, it works fine.
but for better understanding, it's better to check solver status. time limit errors mostly refer to the low limitations.
in this example we have many value more than 300 in time matrix but the maximum time for every vehicle is 300.so there is not a feasible solution for this problem.

Compute optimal route for picking ressources

Here are the rules (Halite 3):
At each turn, you have to take one decision for your ship. You can either move right, move left, move up, move down or stay still (and gather ressoureces).
Moving your ship cost your ship 10% of current's cell available ressources.
Staying still cost you nothing and you pick 25% of current cell available ressources. If a ship doesn't have enought ressources to pay the 10% to move, he is pinned on the cell for this turn (can't go negative).
My goal: The ship starts at the center (0) and should return at this point with atleast (X amount of ressources) in the less turn possible.
How can I compute the most efficient path (round wise) that my ship should take to achieve my goal ?
Here is some code to compute a random map (the map is random every game) :
import random
how_Big = 11
center_Pos = int(how_Big/2) #how_Big must be even
game_Map = [[random.randint(1,500) for i in range(how_Big)] for x in range(how_Big)]
game_Map[center_Pos][center_Pos] = 0
for y in range(how_Big):
print(game_Map[y])
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 172, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
I am not really sure how to start. Is there any known algorithm for this ?
Thanks
Additionnal informations :
The matrix is the available ressources at the beginning on each cell.
Ship begins with 0 ressources.
Ship gains ressources by staying still for a round on a cell
(collecting 25% of the cell's ressources).
Ship spends ressources (of his own cargo) by moving (the cost is 10%
of current cell's available ressources).
The path can be returned in anyway, for example (moves = ["up",
"still", "still", "up", "still", "still", "down", "down"]).
BruteForcing it would be acceptable.
X would be the minimum ammount I want my ship's cargo to have when
returning to center.
Map always has a center Cell
Ressources collected or used are always rounded to nearest Int
Ressources collected by a ship are removed from the cell
Here is an example of what can happen :
move_Dict = {
"up": [-1, 0],
"down": [1, 0],
"right": [0, 1],
"left": [0, -1],
"still": [0, 0],
}
moves = ["up", "still", "still", "up", "still", "still", "down", "down"]
position = [5, 5]
cargo = 0
print("initial map :")
for y in range(how_Big):
print(game_Map[y])
for move in moves:
if move == "still":
ressources_Collected = round(game_Map[position[0]][position[1]] * 0.25)
cargo += ressources_Collected
game_Map[position[0]][position[1]] -= ressources_Collected
else:
ressources_Used = round(game_Map[position[0]][position[1]] * 0.10)
cargo -= ressources_Used
position = [position[0]+move_Dict[move][0], position[1]+move_Dict[move][1]]
print(f"ship is at position {position}, cargo = {cargo}")
for y in range(how_Big):
print(game_Map[y])
Output:
initial map :
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 172, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [4, 5], cargo = 0
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 172, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [4, 5], cargo = 43
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 129, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [4, 5], cargo = 75
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [3, 5], cargo = 65
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 479, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [3, 5], cargo = 185
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 359, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [3, 5], cargo = 275
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 269, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [4, 5], cargo = 248
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 269, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
ship is at position [5, 5], cargo = 238
[110, 179, 97, 467, 347, 336, 368, 298, 107, 84, 123]
[415, 86, 12, 75, 354, 74, 250, 221, 51, 254, 252]
[368, 235, 389, 1, 155, 186, 149, 135, 458, 243, 344]
[391, 480, 485, 358, 416, 269, 270, 354, 203, 436, 146]
[62, 132, 490, 33, 445, 97, 127, 274, 130, 77, 356]
[239, 11, 459, 245, 214, 0, 324, 162, 58, 394, 202]
[241, 395, 46, 78, 191, 384, 203, 191, 56, 474, 237]
[85, 480, 181, 98, 122, 482, 90, 351, 257, 266, 182]
[398, 125, 195, 423, 219, 290, 140, 166, 413, 499, 428]
[213, 367, 142, 471, 141, 407, 382, 229, 332, 455, 53]
[207, 12, 319, 54, 246, 274, 474, 312, 170, 374, 188]
[Finished in 0.1s]

stateful autoencoder in Keras

I'm trying to create a stateful autoencoder model. The goal is to make the autoencoder stateful for each timeseries. The data consists of 10 timeseries and each timeseries has 567 length.
timeseries#1: 451, 318, 404, 199, 225, 158, 357, 298, 339, 155, 135, 239, 306, ....
timeseries#2: 304, 274, 150, 143, 391, 357, 278, 557, 98, 106, 305, 288, 325, ....
...
timeseries#10: 208, 138, 201, 342, 280, 282, 280, 140, 124, 261, 193, .....
My lookback windeow is 28. So I generated the following sequences with 28 timesteps:
[451, 318, 404, 199, 225, 158, 357, 298, 339, 155, 135, 239, 306, .... ]
[318, 404, 199, 225, 158, 357, 298, 339, 155, 135, 239, 306, 56, ....]
[404, 199, 225, 158, 357, 298, 339, 155, 135, 239, 306, 56, 890, ....]
...
[304, 274, 150, 143, 391, 357, 278, 557, 98, 106, 305, 288, 325, ....]
[274, 150, 143, 391, 357, 278, 557, 98, 106, 305, 288, 325, 127, ....]
[150, 143, 391, 357, 278, 557, 98, 106, 305, 288, 325, 127, 798, ....]
...
[208, 138, 201, 342, 280, 282, 280, 140, 124, 261, 193, .....]
[138, 201, 342, 280, 282, 280, 140, 124, 261, 193, 854, .....]
That gives me 539 sequences for each timeseries. What I need to do is to make the LSTMs to be stateful for each of the timeseries and reset the state after seeing all the sequences from a timeseries. Here is the code I have:
batch_size = 35 #(total Number of samples is 5390, and it is dividable by 35)
timesteps = 28
n_features = 1
hunits = 14
RepeatVector(timesteps/hunits = 2)
epochs = 1000
inputEncoder = Input(batch_shape=(35, 28, 1), name='inputEncoder')
outEncoder, c, h = LSTM(14, stateful=True, return_state=True, name='outputEncoder')(inputEncoder)
encoder_model = Model(inputEncoder, outEncoder)
context = RepeatVector(2, name='inputDecoder')(outEncoder)
context_reshaped = Reshape(28, 1), name='ReshapeLayer')(context)
outDecoder = LSTM(1, return_sequences=True, stateful=True, name='decoderLSTM')(context_reshaped)
autoencoder = Model(inputEncoder, outDecoder)
autoencoder.compile(loss='mse', optimizer='rmsprop')
for i in range(epochs):
history = autoencoder.fit(data, data,
validation_split=config['validation_split_ratio'],
shuffle=False,
batch_size=35,
epochs=1,
)
autoencoder.reset_states()
2 questions:
1- I'm getting this error after the first epoch is finished, I wonder how it is happening:
ValueError: Cannot feed value of shape (6, 28, 1) for Tensor u'inputEncoder:0', which has shape '(35, 28, 1)'
2- I don't think that model works as I want. Here it will reset the states after all batches (one epoch) which means after all timeseries are processed. How should I change it to be stateful between timeseries?

The issue is from the validation_split rate!! It is set to a 0.33% and when the splits happens it tries to train on 3611 data samples which is not divisible by my batch_size=35 . Based on this post I could find the proper number, copying from that post:
def quantize_validation_split(validation_split, sample_count, batch_size):
batch_count = sample_count / batch_size
return float(int(batch_count * validation_split)) / batch_count
then you can call model.fit(...,
validation_split=fix_validation_split(0.05, len(X), batch_size)). but
it would be cool if keras did this for you inside fit().
Also, regarding make the autoencoder stateful the way I need: there shouldn't be a reset_state at the end of each epoch!

'list' object is not callable python

numbers = [
951, 402, 984, 651, 360, 69, 408, 319, 601, 485, 980, 507, 725, 547, 544,
615, 83, 165, 141, 501, 263, 617, 865, 575, 219, 390, 984, 592, 236, 105, 942, 941,
386, 462, 47, 418, 907, 344, 236, 375, 823, 566, 597, 978, 328, 615, 953, 345,
399, 162, 758, 219, 918, 237, 412, 566, 826, 248, 866, 950, 626, 949, 687, 217,
815, 67, 104, 58, 512, 24, 892, 894, 767, 553, 81, 379, 843, 831, 445, 742, 717,
958, 609, 842, 451, 688, 753, 854, 685, 93, 857, 440, 380, 126, 721, 328, 753, 470,
743, 527]
count=0
while numbers(count)==527:
print(numbers(count))
count+=1
may in know what is the error in above code?
what is meant by "'list' object is not callable"
please correct the code
thanks.

numbers(count) replace on numbers[count]
count=0
while numbers[count]!=527:
print(numbers(count))
count+=1
If you want print all numbers except last:
for item in numbers[0:-1]:
print(item)

while numbers(count)==527: means that numbers is interpreted as a function with one argument, count. numbers is actually a list object, so when the interpreter tries to call the 'function' it will return the error you're seeing.
You need to use [] for indexing and slicing.
Your revised code would look like this:
numbers = [
951, 402, 984, 651, 360, 69, 408, 319, 601, 485, 980, 507, 725, 547, 544,
615, 83, 165, 141, 501, 263, 617, 865, 575, 219, 390, 984, 592, 236, 105, 942, 941,
386, 462, 47, 418, 907, 344, 236, 375, 823, 566, 597, 978, 328, 615, 953, 345,
399, 162, 758, 219, 918, 237, 412, 566, 826, 248, 866, 950, 626, 949, 687, 217,
815, 67, 104, 58, 512, 24, 892, 894, 767, 553, 81, 379, 843, 831, 445, 742, 717,
958, 609, 842, 451, 688, 753, 854, 685, 93, 857, 440, 380, 126, 721, 328, 753, 470,
743, 527]
count=0
while numbers[count]!=527:
print(numbers[count])
count+=1
This will print every item in numbers up until you get to the first instance of 527.

Resources