Related
In tiptap2, I have two custom extensions that add a class versus a style because I am utilizing tailwindcss, which leverages classes exclusively, not inline styles.
So, the first one adds 'class="text-green-500"' (or whatever) and likewise, 'class="bg-green-500"'. I extend TextStyle in both custom extensions to allow for class versus span. I believe the answer lies in extending textstyle once, but I'm not sure how to go about that and catch both outcomes.
I can't combine the two by having a highlight span and a color span together.
If I take the following:
and then try and say make the "w" a different color, I get:
What I want to achieve is "Howdy" with complete cyan while still able to apply individual font colors within the outer span (or vice-versa).
import { TextStyle } from '#tiptap/extension-text-style';
export const HighlightColorStyle = TextStyle.extend({
parseHTML() {
return [
{
tag: 'span',
getAttrs: (node) => /text|bg-[\w]*-[1-9]00/.test(node.className)
}
];
}
});
export const HighlightColor = Extension.create({
name: 'highlightColor',
addGlobalAttributes() {
return [
{
types: ['textStyle'],
attributes: {
class: {
default: ''
}
}
}
];
},
addCommands() {
return {
setHighlightColor:
(color) =>
({ chain }) => {
console.log('hoadodoadfaf', color);
return chain().setMark('textStyle', { class: color });
},
toggleHighlightColor:
() =>
({ chain }) => {
return chain().toggleMark('textStyle');
},
unsetHighlightColor:
() =>
({ chain }) => {
return chain().setMark('textStyle', { class: null }).removeEmptyTextStyle();
}
};
}
});
and
import { Extension } from '#tiptap/core';
import { TextStyle } from '#tiptap/extension-text-style';
export const TextColorStyle = TextStyle.extend({
parseHTML() {
return [
{
tag: 'span',
getAttrs: node => /text-[\w]*-[1-9]00/.test(node.className)
}
];
}
});
export const TextColor = Extension.create({
name: 'textColor',
addGlobalAttributes() {
return [
{
types: ['textStyle'],
attributes: {
class: {
default: ''
} }
}
];
},
addCommands() {
return {
setTextColor:
color =>
({ chain }) => {
console.log('hoadodoadfaf', color)
return chain().setMark('textStyle', { class: color });
},
toggleTextColor:
() =>
({ chain }) => {
return chain().toggleMark('textStyle');
},
unsetTextColor:
() =>
({ chain }) => {
return chain().setMark('textStyle', { class: null }).removeEmptyTextStyle();
}
};
}
});
You should treat class as object not string, so the multi class can combine, then add a ClassExtension to apply the class to a tag.
The ClassExtension is works like textStyle, except it will apply the class, textStyle dose not apply styles, because styles is an object, applied by individual Extension.
Write a simple classNames extension based on a modified testStyle extension.
const body = {
query: {
geo_shape: {
geometry: {
relation: 'within',
shape: {
type: 'polygon',
coordinates: [$polygon],
},
},
},
},
pit: {
id: "t_yxAwEPZXNyaS1wYzYtMjAxN3IxFjZxU2RBTzNyUXhTUV9XbzhHSk9IZ3cAFjhlclRmRGFLUU5TVHZKNXZReUc3SWcAAAAAAAALmpMWQkNwYmVSeGVRaHU2aDFZZExFRjZXZwEWNnFTZEFPM3JReFNRX1dvOEdKT0hndwAA",
keep_alive: "1m",
},
};
Query fails with search_phase_execution_exception at onBody
Without pit query works fine but it's needed to retrieve more than 10000 hits
Well, using PIT in NodeJS ElasticSearch's client is not clear, or at least is not well documented. You can create a PIT using the client like:
const pitRes = await elastic.openPointInTime({
index: index,
keep_alive: "1m"
});
pit_id = pitRes.body.id;
But there is no way to use that pit_id in the search method, and it's not documented properly :S
BUT, you can use the scroll API as follows:
const scrollSearch = await elastic.helpers.scrollSearch({
index: index,
body: {
"size": 10000,
"query": {
"query_string": {
"fields": [ "vm_ref", "org", "vm" ],
"query": organization + moreQuery
},
"sort": [
{ "utc_date": "desc" }
]
}
}});
And then read the results as follows:
let res = [];
try {
for await (const result of scrollSearch) {
res.push(...result.body.hits.hits);
}
} catch (e) {
console.log(e);
}
I know that's not the exact answer to your question, but I hope it helps ;)
The usage of point-in-time for pagination of search results is now documented in ElasticSearch. You can find more or less detailed explanations here: Paginate search results
I prepared an example that may give an idea about how to implement the workflow, described in the documentation:
async function searchWithPointInTime(cluster, index, chunkSize, keepAlive) {
if (!chunkSize) {
chunkSize = 5000;
}
if (!keepAlive) {
keepAlive = "1m";
}
const client = new Client({ node: cluster });
let pointInTimeId = null;
let searchAfter = null;
try {
// Open point in time
pointInTimeId = (await client.openPointInTime({ index, keep_alive: keepAlive })).body.id;
// Query next chunk of data
while (true) {
const size = remained === null ? chunkSize : Math.min(remained, chunkSize);
const response = await client.search({
// Pay attention: no index here (because it will come from the point-in-time)
body: {
size: chunkSize,
track_total_hits: false, // This will make query faster
query: {
// (1) TODO: put any filter you need here (instead of match_all)
match_all: {},
},
pit: {
id: pointInTimeId,
keep_alive: keepAlive,
},
// Sorting should be by _shard_doc or at least include _shard_doc
sort: [{ _shard_doc: "desc" }],
// The next parameter is very important - it tells Elastic to bring us next portion
...(searchAfter !== null && { search_after: [searchAfter] }),
},
});
const { hits } = response.body.hits;
if (!hits || !hits.length) {
break; // No more data
}
for (hit of hits) {
// (2) TODO: Do whatever you need with results
}
// Check if we done reading the data
if (hits.length < size) {
break; // We finished reading all data
}
// Get next value for the 'search after' position
// by extracting the _shard_doc from the sort key of the last hit
searchAfter = hits[hits.length - 1].sort[0];
}
} catch (ex) {
console.error(ex);
} finally {
// Close point in time
if (pointInTime) {
await client.closePointInTime({ body: { id: pointInTime } });
}
}
}
I wrote a simple Node.js program with a nice menu system facilitated by inquirer.js. However, after selecting an option in the menu and completing some action, the program exits. I need the menu to show again, until I select the Exit [last] option in the menu. I would like to do this using Promise, instead of async/await.
I tried using a function to show the menu and called that function within a forever loop (E.g. while (true) { ... }), but that made the program unusable. I changed that to a for-loop just to observe the problem. Below is the simple program and the resulting output.
PROGRAM
"use strict";
const inquirer = require('inquirer');
const util = require('util')
// Clear the screen
process.stdout.write("\u001b[2J\u001b[0;0H");
const showMenu = () => {
const questions = [
{
type: "list",
name: "action",
message: "What do you want to do?",
choices: [
{ name: "action 1", value: "Action1" },
{ name: "action 2", value: "Action2" },
{ name: "Exit program", value: "quit"}
]
}
];
return inquirer.prompt(questions);
};
const main = () => {
for (let count = 0; count < 3; count++) {
showMenu()
.then(answers => {
if (answers.action === 'Action1') {
return Promise.resolve('hello world');
}
else if (answers.action === 'Action2') {
return new Promise((resolve, reject) => {
inquirer
.prompt([
{
type: 'input',
name: 'secretCode',
message: "Enter a secret code:"
}
])
.then(answers => {
resolve(answers);
})
});
}
else {
console.log('Exiting program.')
process.exit(0);
}
})
.then((data) => { console.log(util.inspect(data, { showHidden: false, depth: null })); })
.catch((error, response) => {
console.error('Error:', error);
});
}
}
main()
OUTPUT
? What do you want to do? (Use arrow keys)
❯ action 1
action 2
Exit program ? What do you want to do? (Use arrow keys)
❯ action 1
action 2
Exit program ? What do you want to do? (Use arrow keys)
❯ action 1
action 2
Exit program (node:983) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 keypress listeners added to [ReadStream]. Use emitter.setMaxListeners() to increase limit
How can I block after the first call to generate the menu, wait for an option to be selected and the corresponding action to complete, and then cycle back to the next iteration of showing the menu?
You can use async/await syntax:
Declare your main function async, and await the returned Promise from inquirer:
const main = async () => {
for (let count = 0; count < 3; count++) {
await showMenu()
.then(answers => {
[...]
}
};
Your code doesn't work as you expect because, in short, the interpreter executes synchronous code before running any callbacks (from promises). As a consequence your synchronous for loop executes before any I/O callbacks are resolved. All calls to showMenu() returns promises which are resolved asynchronously, meaning nothing will be printed, and no inputs will be interpreted until after looping.
Writing await blocks succeeding synchronous code inside an async function, which is what it seems you're trying to do.
Using your code as a starting point, I hacked together my own library for displaying cli menus. It strips away a lot of Inquirer's boilerplate, letting you declare a menu graph/tree concisely.
The main.ts file shows how you use it. You declare a dictionary of MenuPrompts, which you add Menus, Actions and LoopActions to. Each prompt has a key, which other prompts can route to.
// main.ts
import { Menu, Action, MenuPrompt, openMenuPrompt, LoopAction } from "./menus";
// Set of prompts
let prompts = {
menu_1: new MenuPrompt("Menu 1 - This list is ordinal - What would like to do?", 20, true, [
new Menu("Menu 2", "menu_2"),
new LoopAction("Action", () => console.log("Menu 1 action executed")),
new Action("Back", context => context.last),
new Action("Exit", () => process.exit(0)),
]),
menu_2: new MenuPrompt("Menu 2 - This list is NOT ordinal - What would like to do?", 20, false, [
new Menu("Menu 1", "menu_1"),
new LoopAction("Action", () => console.log("Menu 2 action executed")),
new Action("Back", context => context.last),
new Action("Exit", () => process.exit(0)),
]),
};
// Open the "menu_1" prompt
openMenuPrompt("menu_1", prompts);
This is the lib file, which contains types & the function for opening the initial prompt.
// menus.ts
import * as inquirer from "inquirer";
// MAIN FUNCTION
export let openMenuPrompt = async (current: string, prompts: Dict<MenuPrompt>, last?: string): Promise<any> => {
let answer: Answer = (await inquirer.prompt([prompts[current]])).value;
let next = answer.execute({current, last});
if (!next) return;
return await openMenuPrompt(next, prompts, current == next? last : current );
};
// PUBLIC TYPES
export class MenuPrompt {
type = "list";
name = "value";
message: string;
pageSize: number;
choices: Choice[];
constructor(message: string, pageSize: number, isOrdinalList: boolean, choices: Choice[]) {
this.message = message;
this.pageSize = pageSize;
this.choices = choices;
if (isOrdinalList) {
this.choices.forEach((choice, i) => choice.name = `${i + 1}: ${choice.name}`)
}
}
}
export interface Choice {
name: string;
value: Answer;
}
export class Action implements Choice {
name: string;
value: Answer;
constructor(name: string, execute: (context?: MenuContext) => any) {
this.name = name;
this.value = {execute};
}
}
export class LoopAction implements Choice {
name: string;
value: Answer;
constructor(name: string, execute: (context?: MenuContext) => any) {
this.name = name;
this.value = {execute: context => execute(context) ?? context.current};
}
}
export class Menu implements Choice {
name: string;
value: Answer;
constructor(name: string, menuKey: string) {
this.name = name;
this.value = {execute: () => menuKey};
}
}
// INTERNAL TYPES
type Dict<T = any> = {[key: string]: T};
interface Answer {
execute: (context: MenuContext) => any;
}
interface MenuContext {
current: string;
last: string;
}
I have made a job script for web scraping periodically a page and save some information in a MongoDB database. I have tried to get as much performance as i can, and for now i'm able to execute the script each 10s. However, i would like to reduce it even more, with a period between 1-10 seconds if possible. The problem is that when i reduce it, my code throws the following warning and some executions get stacked unresolved:
(node:9472) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit
Is there a way to improve the code?
const $ = require('cheerio');
const MarketModel = require('./models/marketModel');
const mongoose = require('mongoose');
const puppeteer = require('puppeteer');
var schedule = require('node-schedule');
const {
Cluster
} = require('puppeteer-cluster');
//Connection to DataBase:
mongoose.connect('mongodb://localhost:27017/Tradheo', {
useNewUrlParser: true
});
mongoose.connection.on('error', error => console.log(error));
mongoose.Promise = global.Promise;
getMarketData = async () => {
console.log("Web scraping to get market data...")
let markets = []
let marketSpain = {
country: 'Spain',
name: 'IBEX 35',
companies: []
}
let marketGermany = {
country: 'Germany',
name: 'DAX',
companies: []
}
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_PAGE,
maxConcurrency: 2,
});
await cluster.task(async ({
page,
data: url
}) => {
await page.goto({
waitUntil: 'domcontentloaded'
});
await page.setRequestInterception(true);
page.on('request', request => {
if (request.resourceType() === 'document') {
request.continue();
} else {
request.abort();
}
});
const html = await page.content();
if (url === 'https://uk.investing.com/equities/spain') {
console.log('Spain data page content loaded');
$("table[class='genTbl closedTbl crossRatesTbl elpTbl elp30'] > tbody > tr", html).each((i, elem) => {
marketSpain.companies.push({
name: $("td[class='bold left noWrap elp plusIconTd'] > a", html).eq(i).html(),
last: $("td", elem).eq(2).text(),
high: $("td", elem).eq(3).text(),
low: $("td", elem).eq(4).text(),
change: $("td", elem).eq(5).text(),
changePerCent: $("td", elem).eq(6).text(),
volume: $("td", elem).eq(7).text(),
time: $("td", elem).eq(8).text(),
purchase: false,
sale: false
});
});
markets.push(marketSpain);
} else {
console.log('Germany data page content loaded');
$("table[class='genTbl closedTbl crossRatesTbl elpTbl elp30'] > tbody > tr", html).each((i, elem) => {
marketGermany.companies.push({
name: $("td[class='bold left noWrap elp plusIconTd'] > a", html).eq(i).html(),
last: $("td", elem).eq(2).text(),
high: $("td", elem).eq(3).text(),
low: $("td", elem).eq(4).text(),
change: $("td", elem).eq(5).text(),
changePerCent: $("td", elem).eq(6).text(),
volume: $("td", elem).eq(7).text(),
time: $("td", elem).eq(8).text(),
purchase: false,
sale: false
});
});
markets.push(marketGermany);
}
if (markets.length === 2) {
MarketModel.create({
markets,
}, (err) => {
if (err) return handleError(err);
})
console.log("Done!")
}
});
cluster.queue(url1);
cluster.queue(url2);
await cluster.idle();
await cluster.close();
}
var j = schedule.scheduleJob('*/10 * 8-17 * * 1-5', function () {
const now = new Date();
//Checks that time is between 8:30 - 17:35 (schedule of the stock exchange)
if (now.getHours() >= 8 && !(now.getHours() == 8 && now.getMinutes() < 30) && now.getHours() <= 17 && !(now.getHours() == 17 && now.getMinutes() > 35)) {
getMarketData();
}
});
UPDATE: I have added some improvements like setting waitUntil property to 'domcontentloaded' and request interception to avoid waiting for images, and any kind of resources apart from html content, to be loaded. However, seems to be insufficient to achieve the goal.
I Have html+javascript that requests from mongodb database some games(game1,2,3,4,5,6)just simple database with alot of games.
I want to know how via vue.js i can do pagination that per page show 4games.?
const SEARCH = new Vue({
el: '#search',
data: {
query: {
name: '',
max_price:0,
game_category:'',
game_publisher:'',
},
games: [] // current list of games. we re-fill this array after search
},
methods: {
btn_search: function () {
// now we know that this.query is our search critearia object
// so we can do fetch, and will do.
fetch('/search?json=' + JSON.stringify(this.query))
.then((response) => { //as you remember - res is a buffer.
return response.text();
})
.then((text_response) => {
console.log('got response!');
let games_from_server = JSON.parse(text_response);
this.games.splice(0, this.games.length); //it will remove all elemtns from array remove all elemtns from array
// and add games from server one by one.
for (let i = 0; i < games_from_server.length; i++) {
this.games.push(games_from_server[i]);
}
});
console.log(this.query);
}
}
});
console.log('pew?');
If you want to do a client-side pagination you can do it this way:
In your data add currentPage: 1 and gamesPerPage:
data() {
return {
currentPage: 1,
gamesPerPage: 4,
games: []
}
}
then add a computed property paginatedGames which is your games property split into pages, a currentPageGames property which filters games in current page and changePage method which changes your page:
computed: {
paginatedGames() {
let page = 1;
return [].concat.apply(
[],
this.games.map( (game, index) =>
index % this.gamesPerPage ?
[] :
{ page: page++, games: this.games.slice(index, index + this.gamesPerPage)}
)
);
},
currentPageGames() {
let currentPageGames = this.paginatedGames.find(pages => pages.page == this.currentPage);
return currentPageGames ? currentPageGames.games : [];
}
},
methods {
changePage(pageNumber) {
if(pageNumber !== this.currentPage)
this.currentPage = pageNumber;
}
}
Complete example: http://jsfiddle.net/eywraw8t/217989/
However, if your database has lots of games, it might be a better idea to implement a server-side pagination and fetch games only for requested page.