Top 4 Prediction Using Keras Model - keras
I made my own Keras CNN and used the code below to predict. The prediction give all the 143 prediction while I only want the four major classes with the highest percentage.
Code:
preds = model.predict(imgs)
for cls in train_generator.class_indices:
x = preds[0][train_generator.class_indices[cls]]
x_pred = "{:.1%}".format(x)
value = (cls+":"+ x_pred)
print (value)
Prediction:
Acacia_abyssinica:0.0%
Acacia_kirkii:0.0%
Acacia_mearnsii:0.0%
Acacia_melanoxylon:0.0%
Acacia_nilotica:0.0%
Acacia_polyacantha:0.0%
Acacia_senegal:0.0%
Acacia_seyal:0.0%
Acacia_xanthophloea:0.0%
Afrocarpus_falcatus:0.0%
Afzelia_quanzensis:0.0%
Albizia_gummifera:0.0%
Albizia_lebbeck:0.0%
Allanblackia_floribunda:0.0%
Artocarpus_heterophyllus:0.0%
Azadirachta_indica:0.0%
Balanites_aegyptiaca:0.0%
Bersama_abyssinica:0.0%
Bischofia_javanica:0.0%
Brachylaena_huillensis:0.0%
Bridelia_micrantha:0.0%
Calodendron_capensis:0.0%
Calodendrum_capense:0.0%
Casimiroa_edulis:0.0%
Cassipourea_malosana:0.0%
Casuarina_cunninghamiana:0.0%
Casuarina_equisetifolia:4.8%
Catha_edulis:0.0%
Cathium_Keniensis:0.0%
Ceiba_pentandra:39.1%
Celtis_africana:0.0%
Chionanthus_battiscombei:0.0%
Clausena_anisat:0.0%
Clerodendrum_johnstonii:0.0%
Combretum_molle:0.0%
Cordia_africana:0.0%
Cordia_africana_Cordia:0.0%
Cotoneaster_Pannos:0.0%
Croton_macrostachyus:0.0%
Croton_megalocarpus:0.0%
Cupressus_lusitanica:0.0%
Cussonia_Spicata:0.2%
Cussonia_holstii:0.0%
Diospyros_abyssinica:0.0%
Dodonaea_angustifolia:0.0%
Dodonaea_viscosa:0.0%
Dombeya_goetzenii:0.0%
Dombeya_rotundifolia:0.0%
Dombeya_torrida:0.0%
Dovyalis_abyssinica:0.0%
Dovyalis_macrocalyx:0.0%
Drypetes_gerrardii:0.0%
Ehretia_cymosa:0.0%
Ekeber_Capensis:0.0%
Erica_arborea:0.0%
Eriobotrya_japonica:0.0%
Erythrina_abyssinica:0.0%
Eucalyptus_camaldulensis:0.0%
Eucalyptus_globulus:55.9%
Eucalyptus_grandis:0.0%
Eucalyptus_grandis_saligna:0.0%
Eucalyptus_hybrids:0.0%
Eucalyptus_saligna:0.0%
Euclea_divinorum:0.0%
Ficus_indica:0.0%
Ficus_natalensi:0.0%
Ficus_sur:0.0%
Ficus_sycomorus:0.0%
Ficus_thonningii:0.0%
Flacourtia_indica:0.0%
Flacourtiaceae:0.0%
Fraxinus_pennsylvanica:0.0%
Grevillea_robusta:0.0%
Hagenia_abyssinica:0.0%
Jacaranda_mimosifolia:0.0%
Juniperus_procera:0.0%
Kigelia_africana:0.0%
Macaranga_capensis:0.0%
Mangifera_indica:0.0%
Manilkara_Discolor:0.0%
Markhamia_lutea:0.0%
Maytenus_senegalensis:0.0%
Melia_volkensii:0.0%
Meyna_tetraphylla:0.0%
Milicia_excelsa:0.0%
Moringa_Oleifera:0.0%
Murukku_Trichilia_emetica:0.0%
Myrianthus_holstii:0.0%
Newtonia_buchananii:0.0%
Nuxia_congesta:0.0%
Ochna_holstii:0.0%
Ochna_ovata:0.0%
Ocotea_usambarensis:0.0%
Olea_Europaea:0.0%
Olea_africana:0.0%
Olea_capensis:0.0%
Olea_hochstetteri:0.0%
Olea_welwitschii:0.0%
Osyris_lanceolata:0.0%
Persea_americana:0.0%
Pinus_radiata:0.0%
Podocarpus _falcatus:0.0%
Podocarpus_latifolius:0.0%
Polyscias_fulva:0.0%
Polyscias_kikuyuensis:0.0%
Pouteria_adolfi_friedericii:0.0%
Prunus_africana:0.0%
Psidium_guajava:0.0%
Rauvolfia_Vomitoria:0.0%
Rhus_natalensis:0.0%
Rhus_vulgaris:0.0%
Schinus_molle:0.0%
Schrebera_alata:0.0%
Sclerocarya_birrea:0.0%
Scolopia_zeyheri:0.0%
Senna_siamea:0.0%
Sinarundinaria_alpina:0.0%
Solanum_mauritianum:0.0%
Spathodea_campanulata:0.0%
Strychnos_usambare:0.0%
Syzygium_afromontana:0.0%
Syzygium_cordatum:0.0%
Syzygium_cuminii:0.0%
Syzygium_guineense:0.0%
Tamarindus_indica:0.0%
Tarchonanthus_camphoratus:0.0%
Teclea_Nobilis:0.0%
Teclea_simplicifolia:0.0%
Terminalia_brownii:0.0%
Terminalia_mantaly:0.0%
Toddalia_asiatica:0.0%
Trema_Orientalis:0.0%
Trichilia_emetica:0.0%
Trichocladus_ellipticus:0.0%
Trimeria_grandifolia:0.0%
Vangueria_madagascariensis:0.0%
Vepris_nobilis:0.0%
Vepris_simplicifolia:0.0%
Vernonia_auriculifera:0.0%
Vitex_keniensis:0.0%
Warburgia_ugandensis:0.0%
Zanthoxylum_gilletii:0.0%
Mahogany_tree:0.0%
You can just get all your predictions, sort them and take top four
preds = model.predict(imgs)
sorted_preds = []
for cls in train_generator.class_indices:
x = preds[0][train_generator.class_indices[cls]]
x_pred = "{:.1%}".format(x)
sorted_preds.append([x, x_pred, cls])
top_4 = sorted(sorted_preds, reverse=True)[:4]
Related
Numbers of hidden layers and units in AutoKeras dense block
I am training a model with Autokeras. So far my best model is this: structured_data_block_1/normalize:false structured_data_block_1/dense_block_1/use_batchnorm:true structured_data_block_1/dense_block_1/num_layers:2 structured_data_block_1/dense_block_1/units_0:32 structured_data_block_1/dense_block_1/dropout:0 structured_data_block_1/dense_block_1/units_1:32 dense_block_2/use_batchnorm:true dense_block_2/num_layers:2 dense_block_2/units_0:128 dense_block_2/dropout:0 dense_block_2/units_1:16 dense_block_3/use_batchnorm:false dense_block_3/num_layers:1 dense_block_3/units_0:32 dense_block_3/dropout:0 dense_block_3/units_1:32 regression_head_1/dropout:0 optimizer:"adam" learning_rate:0.1 dense_block_2/units_2:32 structured_data_block_1/dense_block_1/units_2:256 dense_block_3/units_2:128 My first dense_block_1 has 2 layers (num_layers:2), how can I have three units / neurons then? It say units_0: 32, units_1: 32 and units_2: 256, this implies to me that I have three layers, so why is num_layers:2? If I would want to recreate the above model in this code, how would I do it properly? input_node = ak.StructuredDataInput() output_node = ak.StructuredDataBlock(categorical_encoding=False, normalize=False)(input_node) output_node = ak.DenseBlock()(output_node) output_node = ak.DenseBlock()(output_node) output_node = ak.RegressionHead()(output_node) Thx for any input
Computing gradient twice for two different losses in Pytorch
I want to compute the gradients twice for two different losses in the same iteration. Code: batch_output0,batch_output1 = get_output_from_model(model=model, data=batch[0]) train_loss0 = loss_fun0(batch_output0, batch_labels0.float().view(-1, 1)) train_loss0.backward() grad0_conv_w = model.conv1.conv1.weight.grad batch_output0,batch_output1 = get_output_from_model(model=model, data=batch[0]) train_loss1 = loss_fun1(batch_output1, batch_labels1.float().view(-1, 1)) train_loss1.backward() grad1_conv_w = model.conv1.conv1.weight.grad Outputs: train_loss0: tensor(0.6950, grad_fn=<BinaryCrossEntropyBackward>) train_loss1: tensor(25.5431, grad_fn=<MseLossBackward>) Grad0: tensor([-2.4883e-05, 3.7842e-05, 1.2635e-04, ..., -1.6413e-04, -1.8419e-04, -1.7884e-04]) Grad1: tensor([-2.4883e-05, 3.7842e-05, 1.2635e-04, ..., -1.6413e-04, -1.8419e-04, -1.7884e-04]) You may note that even though the two losses are quite different, the gradients for the corresponding losses are exactly the same. Please help me to diagnose the problem. Thank you.
BERT zero layer fixed word embeddings [duplicate]
I know that BERT has total vocabulary size of 30522 which contains some words and subwords. I want to get the initial input embeddings of BERT. So, my requirement is to get the table of size [30522, 768] to which I can index by token id to get its embeddings. Where can I get this table?
The BertModels have get_input_embeddings():
import torch
from transformers import BertModel, BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
bert = BertModel.from_pretrained('bert-base-uncased')
token_embedding = {token: bert.get_input_embeddings()(torch.tensor(id)) for token, id in tokenizer.get_vocab().items()}
print(len(token_embedding))
print(token_embedding['[CLS]'])
Output:
30522
tensor([ 1.3630e-02, -2.6490e-02, -2.3503e-02, -7.7876e-03, 8.5892e-03,
-7.6645e-03, -9.8808e-03, 6.0184e-03, 4.6921e-03, -3.0984e-02,
1.8883e-02, -6.0093e-03, -1.6652e-02, 1.1684e-02, -3.6245e-02,
8.3482e-03, -1.2112e-03, 1.0322e-02, 1.6692e-02, -3.0354e-02,
-1.2372e-02, -2.5173e-02, -8.9602e-03, 8.1994e-03, -2.0011e-02,
-1.5901e-02, -3.8394e-03, 1.4241e-03, 7.0500e-03, 1.6092e-03,
-2.7764e-03, 9.4931e-03, -2.2768e-02, 1.9317e-02, -1.3442e-02,
-2.3763e-02, -1.4617e-02, 9.7735e-03, -2.2428e-03, 3.0642e-02,
6.7829e-03, -2.6471e-03, -1.8553e-02, -1.2363e-02, 7.6489e-03,
-2.5461e-03, -3.1498e-01, 6.3761e-03, 4.8914e-02, -7.7636e-03,
6.0919e-02, 2.1346e-02, -3.9741e-02, 2.2853e-01, 2.6502e-02,
-1.0144e-03, -7.8480e-03, -1.9995e-03, 1.7057e-02, -3.3270e-02,
4.5421e-03, 6.1751e-03, -1.0077e-01, -2.0973e-02, -1.4512e-04,
-9.6657e-03, 1.0871e-02, -1.4786e-02, 2.6437e-04, 2.1166e-02,
1.6492e-02, -5.1928e-03, -1.1857e-02, -9.9159e-03, -1.4363e-02,
-1.2405e-02, -1.2973e-02, 2.6778e-02, -1.0986e-02, 1.0572e-02,
-2.5566e-02, 5.2494e-03, 1.5890e-02, -5.1504e-03, -7.5859e-03,
2.0259e-02, -7.0155e-03, 1.6359e-02, 1.7487e-02, 5.4297e-03,
-8.6403e-03, 2.8821e-02, -7.8964e-03, 1.9259e-02, 2.3868e-02,
-4.3472e-03, 5.5662e-02, -2.1940e-02, 4.1779e-03, -5.7216e-03,
2.6712e-02, -5.0371e-03, 2.4923e-02, -1.3429e-02, -8.4337e-03,
9.8188e-02, -1.2940e-03, 1.2865e-02, -1.5930e-03, 3.6437e-03,
1.5569e-02, 1.8620e-02, -9.0643e-03, -1.9740e-02, 1.0530e-02,
-2.7359e-03, -7.5283e-03, 1.1492e-03, 2.6162e-03, -6.2757e-03,
-8.6096e-03, 6.6221e-01, -3.2235e-03, -4.1309e-02, 3.3047e-03,
-2.5040e-03, 1.2838e-04, -6.8073e-03, 6.0291e-03, -9.8468e-03,
8.0641e-03, -1.9815e-03, 2.5801e-02, 5.7429e-03, -1.0712e-02,
2.9176e-02, 5.9414e-03, 2.4795e-02, -1.7887e-02, 7.3183e-01,
1.0964e-02, 5.9942e-03, -4.6157e-02, 4.0131e-02, -9.7481e-03,
-8.9496e-01, 1.6385e-02, -1.9816e-03, 1.4691e-02, -1.9837e-02,
-1.7611e-02, -4.5263e-04, -1.8605e-02, -1.5660e-02, -1.0709e-02,
1.8016e-02, -3.4149e-03, -1.2632e-02, 4.2877e-03, -3.9169e-01,
1.0016e-02, -1.0955e-02, 4.5133e-03, -5.1150e-03, 4.9968e-03,
1.7852e-02, 1.1313e-02, 2.6519e-03, 3.3658e-01, -1.8168e-02,
1.3170e-02, 7.3927e-03, 5.2521e-03, -9.6230e-03, 1.2844e-02,
4.1554e-01, -9.7247e-03, -4.2439e-03, 5.5287e-04, 1.8271e-02,
-1.3889e-03, -2.0502e-03, -8.1946e-03, -6.5979e-06, -7.2764e-04,
-1.4625e-03, -6.9872e-03, -6.9633e-03, -8.0701e-03, 1.9936e-02,
4.8370e-03, 8.6883e-03, -4.9246e-02, -2.0028e-02, 1.4124e-03,
1.0444e-02, -1.1236e-02, -4.4654e-03, -2.0491e-02, -2.7654e-02,
-3.7079e-02, 1.3215e-02, 6.9498e-02, -3.1109e-02, 7.0562e-03,
1.0887e-02, -7.8090e-03, -1.0501e-02, -4.8735e-03, -6.8399e-04,
1.4717e-02, 4.4342e-03, 1.6012e-02, -1.0427e-02, -2.5767e-02,
-2.2699e-01, 8.6569e-02, 2.3453e-02, 4.6362e-02, 3.5609e-03,
2.1353e-02, 2.3703e-02, -2.0252e-02, 2.1580e-02, 7.2652e-03,
2.0933e-01, 1.2108e-02, 1.0869e-02, 7.0568e-03, -3.1132e-02,
2.0505e-02, 3.2248e-03, -2.2724e-03, 5.5342e-03, 3.0563e-03,
1.9542e-02, 1.2827e-03, 1.5952e-02, -1.5458e-02, -3.8455e-03,
-4.9417e-03, -1.0446e-02, 7.0516e-03, 2.2467e-03, -9.3643e-03,
1.9163e-02, 1.4239e-02, -1.5816e-02, 8.7413e-03, 2.4737e-02,
-7.3777e-03, -4.0975e-02, 9.4948e-03, 1.4700e-02, 2.6819e-02,
1.0706e-02, 1.0621e-02, -7.1816e-03, -8.5402e-03, 1.2261e-02,
-4.8679e-03, -9.6136e-03, 7.8765e-04, 3.8504e-02, -7.7485e-03,
-6.5018e-03, 3.4352e-03, 2.2931e-04, 5.7456e-03, -4.8441e-03,
-9.0898e-03, 8.6298e-03, 5.4740e-03, 2.2274e-02, -2.1218e-02,
-2.6795e-02, -3.5337e-03, 1.0785e-02, 1.2475e-02, -6.1160e-03,
1.0729e-02, -9.7955e-03, 1.8543e-02, -6.0488e-03, -4.5744e-03,
2.7089e-03, 1.5632e-02, -1.2928e-02, -3.0778e-03, -1.0325e-02,
-7.9550e-03, -6.3065e-02, 2.1062e-02, -6.6717e-03, 8.4616e-03,
1.4475e-02, 1.1477e-01, -2.2838e-02, -3.7491e-02, -3.6218e-02,
-3.1994e-02, -8.9252e-03, 3.1720e-02, -1.1260e-02, -1.2980e-01,
-1.0315e-03, -4.7242e-03, -2.0092e-02, -9.4521e-01, -2.2178e-02,
-4.4297e-04, 1.9711e-02, 3.3402e-02, -1.0513e-02, 1.4492e-02,
-1.9697e-02, -9.8452e-03, -1.7347e-02, 2.3472e-02, 7.6570e-02,
1.9504e-02, 9.3617e-03, 8.2672e-03, -1.0471e-02, -1.9932e-03,
2.0000e-02, 2.0485e-02, 1.0977e-02, 1.7720e-02, 1.3532e-02,
7.3682e-03, 3.4906e-04, 1.8772e-03, 1.9976e-02, -3.2041e-02,
-8.9169e-03, 1.2900e-02, -1.3331e-02, 6.6207e-03, -5.7063e-03,
-1.1482e-02, 8.3907e-03, -6.4162e-03, 1.5816e-02, 7.8921e-03,
4.4177e-03, 2.2568e-02, 1.0239e-02, -3.0194e-04, 1.3294e-02,
-2.1606e-02, 3.8832e-03, 2.4475e-02, 4.3808e-02, -2.1031e-03,
-1.2163e-02, -4.0786e-02, 1.5565e-02, 1.4750e-02, 1.6645e-02,
2.8083e-02, 1.8920e-03, -1.4733e-04, -2.6208e-02, 2.3780e-02,
1.8657e-04, -2.2931e-03, 3.0334e-03, -1.7294e-02, -2.3001e-02,
8.6004e-03, -3.3497e-02, 2.5660e-02, -1.9225e-02, -2.7186e-02,
-2.1020e-02, -3.5213e-02, -1.8228e-03, -8.2840e-03, 1.1212e-02,
1.0387e-02, -3.4194e-01, -1.9705e-03, 1.1558e-02, 5.1976e-03,
7.4498e-03, 5.7142e-03, 2.8401e-02, -7.7551e-03, 1.0682e-02,
-1.2657e-02, -1.8065e-02, 2.6681e-03, 3.3947e-03, -4.5565e-02,
-2.1170e-02, -1.7830e-02, 3.4679e-03, -2.2051e-02, -5.4176e-03,
-1.1517e-02, -3.4155e-02, -3.0335e-03, -1.3915e-02, 6.2173e-03,
-1.1101e-02, -1.5308e-02, 9.2188e-03, -7.5665e-03, 6.5685e-03,
8.0935e-03, 3.1139e-03, -5.5047e-03, -3.1347e-02, 2.2140e-02,
1.0865e-02, -2.7849e-02, -4.9580e-03, 1.8804e-03, 1.0007e-01,
-1.8013e-03, -4.8792e-03, 1.5534e-02, -2.0179e-02, -1.2351e-02,
-1.3871e-02, 1.1439e-02, -9.0208e-03, 1.2580e-02, -2.5973e-02,
-2.0398e-02, -1.9464e-03, 4.3189e-03, 2.0707e-02, 5.0029e-03,
-1.0679e-02, 1.2298e-02, 1.0269e-02, 2.2228e-02, 2.9754e-02,
-2.6392e-03, 1.9286e-02, -1.5137e-02, 2.1914e-01, 1.3030e-02,
-7.4460e-03, -9.6818e-04, 2.9736e-02, 9.8722e-03, -5.6688e-03,
4.2518e-03, 1.8941e-02, -6.3909e-03, 8.0590e-03, -6.7893e-03,
6.0878e-03, -5.3970e-03, 7.5776e-04, 1.1374e-03, -5.0035e-03,
-1.6159e-03, 1.6764e-02, 9.1251e-03, 1.3020e-02, -1.0368e-02,
2.2141e-02, -2.5411e-03, -1.5227e-02, 2.3444e-02, 8.4076e-04,
-1.1465e-01, 2.7017e-03, -4.4961e-03, 2.9762e-04, -3.9612e-03,
8.9038e-05, 2.8683e-02, 5.0068e-03, 1.6509e-02, 7.8983e-04,
5.7728e-03, 3.2685e-02, -1.0457e-01, 1.2989e-02, 1.1278e-02,
1.1943e-02, 1.5258e-02, -6.2411e-04, 1.0682e-04, 1.2087e-02,
7.2984e-03, 2.7758e-02, 1.7572e-02, -6.0345e-03, 1.7211e-02,
1.4121e-02, 6.4663e-02, 9.1813e-03, 3.2555e-03, -3.2667e-02,
2.9132e-02, -1.7770e-02, 1.5302e-03, -2.9944e-02, -2.0706e-02,
-3.6528e-03, -1.5497e-02, 1.5223e-02, -1.4751e-02, -2.2381e-02,
6.9636e-03, -8.0838e-03, -2.4583e-03, -2.0677e-02, 8.8132e-03,
-6.9554e-04, 1.6965e-02, 1.8535e-01, 3.5843e-04, 1.0812e-02,
-4.2391e-03, 8.1779e-03, 3.4144e-02, -1.8996e-03, 2.9939e-03,
3.6898e-04, -1.0144e-02, -5.7416e-03, -5.7676e-03, 1.7565e-01,
-1.5793e-03, -2.6617e-02, -1.2572e-02, 3.0421e-04, -1.2132e-02,
-1.4168e-02, 1.2154e-02, 8.4700e-03, -1.6284e-02, 2.6983e-03,
-6.8554e-03, 2.7829e-01, 2.4060e-02, 1.1130e-02, 7.6095e-04,
3.1341e-01, 2.1668e-02, 1.0277e-02, -3.0065e-02, -8.3565e-03,
5.2488e-03, -1.1287e-02, -1.8266e-02, 1.1814e-02, 1.2662e-02,
2.9036e-04, 7.0254e-04, -1.4084e-02, 1.2925e-02, 3.9504e-03,
-7.9568e-03, 3.2794e-02, 7.3839e-03, 2.4609e-02, 9.6109e-03,
-8.7206e-03, 9.2571e-03, -3.5850e-03, -8.9996e-03, 2.3120e-03,
-1.8475e-02, -1.9610e-02, 1.1994e-02, 6.7156e-03, 1.9903e-02,
3.0703e-02, -4.9538e-03, -6.1673e-02, -6.4986e-03, -2.1317e-02,
-3.3650e-03, 2.3200e-03, -6.2224e-03, 3.7458e-03, 1.1542e-02,
-1.0181e-02, -8.4711e-03, 1.1603e-02, -5.6247e-03, -1.0220e-02,
-8.6501e-04, -1.2285e-02, -8.7487e-03, -1.1265e-02, 1.6322e-02,
1.5160e-02, 1.8882e-02, 5.1557e-03, -8.8616e-03, 4.2153e-03,
-1.9450e-02, -8.7365e-03, -9.7867e-03, 1.1667e-02, 5.0613e-03,
2.8221e-03, -7.1795e-03, 9.3306e-03, -4.9663e-02, 1.7708e-02,
-2.0959e-02, -3.3989e-02, 2.2581e-03, 5.1748e-03, -1.0133e-01,
2.1052e-03, 5.5644e-03, 1.3607e-03, 8.8388e-03, 1.0244e-02,
-3.8072e-03, 5.9209e-03, 6.7993e-03, 1.1594e-02, -1.1802e-02,
-2.4233e-03, -5.1504e-03, -1.1903e-02, 1.4075e-02, -4.0701e-03,
-2.9465e-02, -1.7579e-03, 4.3654e-03, 1.0429e-02, 3.7096e-02,
8.6493e-03, 1.5871e-02, 1.8034e-02, -3.2165e-03, -2.1941e-02,
2.6274e-02, -7.6941e-03, -5.9618e-03, -1.4179e-02, 8.0281e-03,
1.1293e-02, -6.6936e-05, 1.2899e-02, 1.0056e-02, -6.3919e-04,
2.0299e-02, 3.1528e-03, -4.8988e-03, 3.2754e-03, -1.1003e-01,
1.8414e-02, 2.2272e-03, -2.2185e-02, -4.8672e-03, 1.9643e-03,
3.0928e-02, -8.9599e-03, -1.1446e-02, -1.3794e-02, 7.1943e-03,
-5.8965e-03, 2.2605e-03, -2.6114e-02, -5.6616e-03, 6.5073e-03,
9.2219e-02, -6.7243e-03, 4.4427e-04, 7.2846e-03, -1.1021e-02,
7.8802e-04, -3.8878e-03, 1.0489e-02, 9.2883e-03, 1.8895e-02,
2.1808e-02, 6.2590e-04, -2.6519e-02, 7.0343e-04, -2.9067e-02,
-9.1515e-03, 1.0418e-03, 8.3222e-03, -8.7548e-03, -2.0637e-03,
-1.1450e-02, -8.8985e-04, -4.4062e-03, 2.3629e-02, -2.7221e-02,
3.2008e-02, 6.6325e-03, -1.1302e-02, -1.0138e-03, -1.6902e-01,
-8.4473e-03, 2.8536e-02, 1.4117e-03, -1.2136e-02, -1.4781e-02,
4.9960e-03, 3.3916e-02, 5.2710e-03, 1.7382e-02, -4.6315e-03,
1.1680e-02, -9.1395e-03, 1.8310e-02, 1.2321e-02, -2.4871e-02,
1.1535e-02, 5.0308e-03, 5.5028e-03, -7.2184e-03, -5.5210e-03,
1.7085e-02, 5.7236e-03, 1.7463e-03, 1.9969e-03, 6.1670e-03,
2.9347e-03, 1.3946e-02, -1.9984e-03, 1.0091e-02, 1.0388e-03,
-6.1902e-03, 3.0905e-02, 6.6038e-03, -9.1223e-02, -1.8411e-02,
5.4185e-03, 2.4396e-02, 1.5696e-02, -1.2742e-02, 1.8126e-02,
-2.6138e-02, 1.1170e-02, -1.3058e-02, -1.9386e-02, -5.9828e-03,
1.9176e-02, 1.9962e-03, -2.1538e-03, 3.3003e-02, 1.8407e-02,
-5.9498e-03, -3.2533e-03, -1.8917e-02, -1.5897e-02, -4.7057e-03,
5.4162e-03, -3.0037e-02, 8.6773e-03, -1.7942e-03, 6.6826e-03,
-1.1929e-02, -1.4076e-02, 1.6709e-02, 1.6860e-03, -3.3842e-03,
8.6805e-03, 7.1340e-03, 1.5147e-02], grad_fn=<EmbeddingBackward>)
To get context-sensitive word embedding for given input sentence/text, here is the code,
import numpy as np
import torch
from transformers import AutoTokenizer, AutoModel
def get_word_idx(sent: str, word: str):
return sent.split(" ").index(word)
def get_hidden_states(encoded, token_ids_word, model, layers):
"""Push input IDs through model. Stack and sum `layers` (last four by default).
Select only those subword token outputs that belong to our word of interest
and average them."""
with torch.no_grad():
output = model(**encoded)
# Get all hidden states
states = output.hidden_states
# Stack and sum all requested layers
output = torch.stack([states[i] for i in layers]).sum(0).squeeze()
# Only select the tokens that constitute the requested word
word_tokens_output = output[token_ids_word]
return word_tokens_output.mean(dim=0)
def get_word_vector(sent, idx, tokenizer, model, layers):
"""Get a word vector by first tokenizing the input sentence, getting all token idxs
that make up the word of interest, and then `get_hidden_states`."""
encoded = tokenizer.encode_plus(sent, return_tensors="pt")
# get all token idxs that belong to the word of interest
token_ids_word = np.where(np.array(encoded.word_ids()) == idx)
return get_hidden_states(encoded, token_ids_word, model, layers)
def main(layers=None):
# Use last four layers by default
layers = [-4, -3, -2, -1] if layers is None else layers
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
model = AutoModel.from_pretrained("bert-base-cased", output_hidden_states=True)
sent = "I like cookies ."
idx = get_word_idx(sent, "cookies")
word_embedding = get_word_vector(sent, idx, tokenizer, model, layers)
return word_embedding
if __name__ == '__main__':
main()
More details can be found here.
Get slope and itercept from a matched linear regression model in scikit-learn
I have a simple model from sklearn import linear_model x =[6.1101, 5.5277, 8.5186, 7.0032, 5.8598, 8.3829, 7.4764, 8.5781, 6.4862, 5.0546, 5.7107, 14.164, 5.734, 8.4084, 5.6407, 5.3794, 6.3654, 5.1301, 6.4296, 7.0708, 6.1891, 20.27, 5.4901, 6.3261, 5.5649, 18.945, 12.828, 10.957, 13.176, 22.203, 5.2524, 6.5894, 9.2482, 5.8918, 8.2111, 7.9334, 8.0959, 5.6063, 12.836, 6.3534, 5.4069, 6.8825, 11.708, 5.7737, 7.8247, 7.0931, 5.0702, 5.8014, 11.7, 5.5416, 7.5402, 5.3077, 7.4239, 7.6031, 6.3328, 6.3589, 6.2742, 5.6397, 9.3102, 9.4536, 8.8254, 5.1793, 21.279, 14.908, 18.959, 7.2182, 8.2951, 10.236, 5.4994, 20.341, 10.136, 7.3345, 6.0062, 7.2259, 5.0269, 6.5479, 7.5386, 5.0365, 10.274, 5.1077, 5.7292, 5.1884, 6.3557, 9.7687, 6.5159, 8.5172, 9.1802, 6.002, 5.5204, 5.0594, 5.7077, 7.6366, 5.8707, 5.3054, 8.2934, 13.394, 5.4369] y = [17.592, 9.1302, 13.662, 11.854, 6.8233, 11.886, 4.3483, 12, 6.5987, 3.8166, 3.2522, 15.505, 3.1551, 7.2258, 0.71618, 3.5129, 5.3048, 0.56077, 3.6518, 5.3893, 3.1386, 21.767, 4.263, 5.1875, 3.0825, 22.638, 13.501, 7.0467, 14.692, 24.147, -1.22, 5.9966, 12.134, 1.8495, 6.5426, 4.5623, 4.1164, 3.3928, 10.117, 5.4974, 0.55657, 3.9115, 5.3854, 2.4406, 6.7318, 1.0463, 5.1337, 1.844, 8.0043, 1.0179, 6.7504, 1.8396, 4.2885, 4.9981, 1.4233, -1.4211, 2.4756, 4.6042, 3.9624, 5.4141, 5.1694, -0.74279, 17.929, 12.054, 17.054, 4.8852, 5.7442, 7.7754, 1.0173, 20.992, 6.6799, 4.0259, 1.2784, 3.3411, -2.6807, 0.29678, 3.8845, 5.7014, 6.7526, 2.0576, 0.47953, 0.20421, 0.67861, 7.5435, 5.3436, 4.2415, 6.7981, 0.92695, 0.152, 2.8214, 1.8451, 4.2959, 7.2029, 1.9869, 0.14454, 9.0551, 0.61705] # Create linear regression object regr = linear_model.LinearRegression() # Train the model using the training sets regr.fit([x], [y]) #where x and y are arrays of values I need to get the slope and intercept. I tried regr.intercept_ but it returns a large array of numbers that I do not understand what it is.
You should do something like this to get the correct results: import numpy as np regr.fit(np.array(x).reshape(-1, 1), np.array(y).reshape(-1, 1))
How to record the value of a variable within odeint?
I would like to know if there is a way to record the value of a specific variable within the function of integration, without having to print it within the definition of the function, which in many cases, due to the algorithm of prediction-correction, lead to more or less values than the final vector returned by the function?
Example let's try with this code:
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
def essai(y, t):
a = y[0]
c1 = a
a = c1 / a**2
return [a]
# Solving
essai0 = [10]
t = np.linspace(0, 2000, 10)
y = odeint(essai, essai0, t)
a = y[:, 0]
# Graphs
fig, ax = plt.subplots()
ax.plot(t, a, 'k--', label='a')
legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
legend.get_frame().set_facecolor('#FFFCCC') # 00FFCC
plt.xlabel('x')
plt.ylabel('y')
plt.title('y vs x')
plt.show()
I would like to record the values of c1 which depends on a. What should I do?
If I print, it I get (because of pred-corr algorithm):
10.0
10.001203411814794
10.00120326701222
10.002406534059283
10.00240638930896
10.031168251789499
10.03116843523562
10.059847893733858
10.059848247411573
10.088446178306066
10.088446526968276
10.178981333917179
10.1789826635142
10.26872274187664
10.268720875457465
10.251795853148066
10.251794757670828
10.324093402400061
10.324093338929458
10.395889284010963
10.395889126663482
10.467192620394076
10.467192470562162
10.60836217080531
10.608361512785885
10.747675991273601
10.747676529983982
10.885208084361661
10.88520861500753
11.021024408838219
11.021024559158226
11.15518691385528
11.15518704871583
11.389028983440005
11.389029612664437
11.618166387462095
11.618166372845774
11.842871925632974
11.842870666797078
12.063390475531826
12.0633901508557
12.279950446401756
12.279950250452782
12.492757035192547
12.492756877414479
12.790475076345272
12.79047467718475
13.081418818481728
13.081418595295522
13.366029970579808
13.366030900758636
13.644707388512776
13.644707798536366
13.917805722870085
13.917805853240296
14.185647189512732
14.185647276304193
14.448524340486092
14.44852440612534
14.849045554474056
14.849045812160185
15.239043242348172
15.239044113472564
15.619306858637934
15.619307570817467
15.990530200625596
15.990530706701604
16.353328829257094
16.35332918566708
16.70825155213741
16.708251810028536
17.055790075751844
17.055790265472186
17.52054793291328
17.520548366986496
17.97329155702487
17.97329263337524
18.414908470097206
18.41490919183692
18.84617978510828
18.846180323693773
19.26780035288661
19.26780072790131
19.68039039537204
19.680390669145883
20.084506483562638
20.084506685872917
20.63204921728682
20.632049705019547
21.165431430483114
21.16543268212929
21.685699626883885
21.685700483180575
22.193774842932424
22.193775478119036
22.69047628806277
22.69047673120133
23.176535191516802
23.1765355148269
23.652607704971896
23.652607943862492
24.296731084127696
24.296731656936466
24.92421316694978
24.924214631653445
25.536282592848192
25.536283593100098
26.134020839947766
26.134021582629195
26.718389929663125
26.718390447872228
27.290248649274574
27.290249027491374
27.8503676838429
27.85036796338048
28.60821935477876
28.608220025227006
29.346505899333515
29.346507613905608
30.066670806260635
30.066671977520553
30.769984796557875
30.769985666417984
31.457578314647648
31.457578921761066
32.13046057231114
32.13046101551341
32.78953730742519
32.789537635058444
33.68118868621462
33.68118947182226
34.54983545122736
34.549837459883506
35.39717380841791
35.397175180698845
36.22469707822626
36.224698097642104
37.033733817898586
37.03373452954837
37.82547018189015
37.82547070150822
38.60097077071101
38.60097115490064
39.65004988104156
39.650050802111195
40.67207751401193
40.67207986867377
41.669047220267416
41.66904882908885
42.64271422854618
42.64271542393563
43.594640193459966
43.59464102811222
44.52621945824691
44.52622006777859
45.43870353935591
45.438703990091476
46.67300975177773
46.673010832232926
47.87550305124021
47.87550581301012
49.04852683447106
49.04852872160157
50.194144483083306
50.19414588551954
51.3141919066777
51.31419288605143
52.41030839692969
52.41030911225109
53.48396538985435
53.483965918885744
54.9362075454971
54.93620881348237
56.35103457439806
56.35103781516747
57.73120149400896
57.731203708595864
59.07913425147381
59.07913589751868
60.39699143853227
60.39699258818307
61.68670054765226
61.68670138744394
62.94999176730058
62.949992388453296
64.65865496068966
64.65865644932029
Which is much more values than I may expect with t = np.linspace(0, 2000, 10) which divide the intervale of time in tenth of 200.
I have thought to this problem for a long time without find a really good way to do it and I would be delighted to know how to bypass this problem.
There is no relation between the evaluation points of the ODE function in the internal solver steps and the requested sample points of the solution for the output. Moreover, the evaluation points can deviate from the solution trajectory with some error of an order lower than the order of the integration method. The easiest way to do what you want in a structured fashion is to define the c1 function as a separate function and then to call it on the results def c1_func(y): return y[0] def essai(y, t): a = y[0] c1 = c1_func(y) a = c1 / a**2 return [a] ... y = odeint(... c1_val = c1_func(y.T) plt.plot(x, c1_val) or so.