why run "python run_squad.py" doesn't work? - nlp

I want fine tune on squad with huggingface run_squad.py, but meet the following question:
1, when I use "--do_train" without "True" as following code, after 20 minutes runing,there is no models in output_dir:
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
2, when I use "--do_train=True" as following code, the error message is "run_squad.py: error: argument --do_train: ignored explicit argument 'True'":
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train=True \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
3, when I use "--do_train True" as following code, the error message is "run_squad.py: error: unrecognized arguments: True":
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train True \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
I run code in colab with GPU: Tesla P100-PCIE-16GB
Judging by the running time, I think the code didn't through training process, but I don't know how to set parameters in order to let training go.what should I do?

Related

kubernetes config map - syntax error: unterminated quoted string

I am getting below error when trying to attach shell script as config map.
I am not sure what's the issue because script work without adding in config map
It shows error is on the line 58
Which is not even there.
Any help will be really appreciated.
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.metadata.name }}-micro
data:
micro-integrator.sh: |
#!/bin/sh
# micro-integrator.sh
while [ "$status" = "$START_EXIT_STATUS" ]
do
$JAVACMD \
-Xbootclasspath/a:"$CARBON_XBOOTCLASSPATH" \
$JVM_MEM_OPTS \
-XX:+HeapDumpOnOutOfMemoryError \
-XX:HeapDumpPath="$CARBON_HOME/repository/logs/heap-dump.hprof" \
$JAVA_OPTS \
-Dcom.sun.management.jmxremote \
-classpath "$CARBON_CLASSPATH" \
-Djava.io.tmpdir="$CARBON_HOME/tmp" \
-Dcatalina.base="$CARBON_HOME/wso2/lib/tomcat" \
-Dwso2.server.standalone=true \
-Dcarbon.registry.root=/ \
-Djava.command="$JAVACMD" \
-Dqpid.conf="/conf/advanced/" \
$JAVA_VER_BASED_OPTS \
-Dcarbon.home="$CARBON_HOME" \
-Dlogger.server.name="micro-integrator" \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Dcarbon.config.dir.path="$CARBON_HOME/conf" \
-Dcarbon.repository.dir.path="$CARBON_HOME/repository" \
-Dcarbon.components.dir.path="$CARBON_HOME/wso2/components" \
-Dcarbon.dropins.dir.path="$CARBON_HOME/dropins" \
-Dcarbon.external.lib.dir.path="$CARBON_HOME/lib" \
-Dcarbon.patches.dir.path="$CARBON_HOME/patches" \
-Dcarbon.internal.lib.dir.path="$CARBON_HOME/wso2/lib" \
-Dcom.atomikos.icatch.hide_init_file_path=true \
-Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false \
-Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true \
-Dcom.sun.jndi.ldap.connect.pool.authentication=simple \
-Dcom.sun.jndi.ldap.connect.pool.timeout=3000 \
-Dorg.terracotta.quartz.skipUpdateCheck=true \
-Djava.security.egd=file:/dev/./urandom \
-Dfile.encoding=UTF8 \
-Djava.net.preferIPv4Stack=true \
-DNonRegistryMode=true \
-DNonUserCoreMode=true \
-Dcom.ibm.cacheLocalHost=true \
-Dcarbon.use.registry.repo=false \
-DworkerNode=false \
-Dorg.apache.cxf.io.CachedOutputStream.Threshold=104857600 \
-DavoidConfigHashRead=true \
-Dproperties.file.path=default \
-DenableReadinessProbe=true \
-DenableManagementApi=true \
$NODE_PARAMS \
-Dorg.apache.activemq.SERIALIZABLE_PACKAGES="*" \
org.wso2.micro.integrator.bootstrap.Bootstrap $*
status="$?"
done

How to create user for connect to database

ERROR :
[FATAL] [DBT-05509] Failed to connect to the specified database (cdb21).
CAUSE: OS Authentication might be disabled for this database (cdb21).
ACTION: Specify a valid sysdba user name and password to connect to the database.
First step:
./runInstaller -silent -responseFile /scratch/app/user/product/21.0.0/dbhome_1/install/response/db_install.rsp \
oracle.install.option=INSTALL_DB_SWONLY \
UNIX_GROUP_NAME=oinstall \
ORACLE_BASE=/scratch/app/user \
INVENTORY_LOCATION=/scratch/app/oraInventory \
SELECTED_LANGUAGES=en \
oracle.install.db.InstallEdition=EE \
oracle.install.db.isCustomInstall=false \
oracle.install.db.OSDBA_GROUP=oinstall \
oracle.install.db.OSBACKUPDBA_GROUP=oinstall \
oracle.install.db.OSDGDBA_GROUP=oinstall \
oracle.install.db.OSKMDBA_GROUP=oinstall \
oracle.install.db.OSRACDBA_GROUP=oinstall \
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false \
DECLINE_SECURITY_UPDATES=true
Second step:
dbca -silent -createDatabase \
-templateName General_Purpose.dbc \
-gdbname cdb21 \
-sid cdb21 \
-responseFile NO_VALUE \
-characterSet AL32UTF8 \
-sysPassword Welcome1 \
-systemPassword Welcome1 \
-createAsContainerDatabase true \
-numberOfPDBs 1 \
-pdbName pdb21 \
-pdbAdminPassword Welcome1 \
-databaseType MULTIPURPOSE \
-memoryMgmtType auto_sga \
-totalMemory 4096 \
-storageType FS \
-datafileDestination /scratch/oradata/ \
-emConfiguration NONE \
-ignorePreReqs
Start The service using :
lsnrctl start
Then :
startup

How to solve this problem in python (Python-3.6.3/Modules/binascii.gcda:Cannot open)

tensorflow 1.14.0, Ubuntu 16.04
when I try to Run training file it give me this error:
profiling:/home/shalaw/Python-3.6.3/build/temp.linux-x86_64-3.6/home/shalaw/Python-3.6.3/Modules/arraymodule.gcda:Cannot open
profiling:/home/shalaw/Python-3.6.3/build/temp.linux-x86_64-3.6/home/shalaw/Python-3.6.3/Modules/_ssl.gcda:Cannot
here is my code:
python3.6 main.py \
--output_dir /home/shalaw/handwritten-data/experiment_sign_semi_reg \
--summary_dir /home/shalaw/handwritten-data/experiment_sign_semi_reg/log/ \
--mode train \
--train_dir /home/shalaw/handwritten-data/signatures/full_org \
--val_dir /home/shalaw/handwritten-data/signatures/val \
--val_dataset_name kaggle_signature \
--learning_rate 0.0001 \
--loss semi-hard \
--decay_step 50000 \
--decay_rate 0.1 \
--stair True \
--beta 0.9 \
--loss_margin 0.5 \
--max_iter 200000
also you can find full code here:
https://github.com/rmalav15/signature-recognition

Sphinx: Unable to open Sphinx/mixture_weights

Im executing this command:
./bw \
-hmmdir Sphinx \
-moddeffn cmusphinx-en-us-ptm-5.2/mdef \
-ts2cbfn .ptm. \
-feat 1s_c_d_dd \
-svspec 0-12/13-25/26-38 \
-cmn current \
-agc none \
-dictfn cmudict-en-us.dict \
-ctlfn arctic20.fileids \
-lsnfn arctic20.transcription \
-accumdir .
And Im getting this error message:
ERROR: "s3io.c", line 260: Unable to open Sphinx/mixture_weights for reading: No such file or directory
The "mixture_weights" file is in the same Folder as the mdef file. What am I doing wrong?
THanks in advance.
You specify wrong hmmdir. Try this
./bw \
-hmmdir cmusphinx-en-us-ptm-5.2 \
-moddeffn cmusphinx-en-us-ptm-5.2/mdef \
-ts2cbfn .ptm. \
-feat 1s_c_d_dd \
-svspec 0-12/13-25/26-38 \
-cmn current \
-agc none \
-dictfn cmudict-en-us.dict \
-ctlfn arctic20.fileids \
-lsnfn arctic20.transcription \
-accumdir .

Make error while ARM cross compiling an SQLite test fixture

I am trying to create a testfixture for SQLite for ARM from a Linux pc. However, I am unable to compile as I am prompted with errors.
I get the following error,
Make: *** No rule to make target `armv7l-timesys-linux-gnueabi-gcc', \
needed by `testfixture'. Stop.
Below is the make file I am using.
#!/usr/make
#
# Makefile for SQLITE
#
# Source code to the test files.
#
# set up compiler and options
TOP = .
INCLUDES = -I"/home/bkrishnan/Downloads/sqlite-amalgamation/testfixture"
CC = armv7l-timesys-linux-gnueabi-gcc
CFLAGS = -g $(INCLUDES) -DSQLITE_PRIVATE="" -DSQLITE_TEST=1 -DTCLSH=1 -D_FILE_OFFSET_BITS=64 -DSQLITE_CORE
# library files.
LIB = libtcl8.5.so
#-----File Dependencies----------------------
# Source files.
SRC = \
$(TOP)/tclsqlite.c \
$(TOP)/sqlite3.c \
$(TOP)/shell.c
# Source OBJ files.
SRCOBJ = \
tclsqlite.o \
sqlite3.o \
shell.o
# Test source files.
TESTSRC = \
$(TOP)/test_async.c \
$(TOP)/test_autoext.c \
$(TOP)/test_backup.c \
$(TOP)/test_btree.c \
$(TOP)/test_config.c \
$(TOP)/test_demovfs.c \
$(TOP)/test_devsym.c \
$(TOP)/test_func.c \
$(TOP)/test_hexio.c \
$(TOP)/test_init.c \
$(TOP)/test_intarray.c \
$(TOP)/test_journal.c \
$(TOP)/test_malloc.c \
$(TOP)/test_mutex.c \
$(TOP)/test_onefile.c \
$(TOP)/test_osinst.c \
$(TOP)/test_pcache.c \
$(TOP)/test_schema.c \
$(TOP)/test_server.c \
$(TOP)/test_stat.c \
$(TOP)/test_tclvar.c \
$(TOP)/test_thread.c \
$(TOP)/test_vfs.c \
$(TOP)/test_wsd.c \
$(TOP)/test1.c \
$(TOP)/test2.c \
$(TOP)/test3.c \
$(TOP)/test4.c \
$(TOP)/test5.c \
$(TOP)/test6.c \
$(TOP)/test7.c \
$(TOP)/test8.c \
$(TOP)/test9.c \
$(TOP)/test_fuzzer.c \
$(TOP)/test_multiplex.c \
$(TOP)/test_quota.c \
$(TOP)/test_rtree.c \
$(TOP)/test_superlock.c \
$(TOP)/test_syscall.c \
$(TOP)/test_wholenumber.c \
$(TOP)/test_loadext.c \
$(TOP)/test_spellfix.c \
$(TOP)/test_vfstrace.c \
$(TOP)/fts3_term.c \
$(TOP)/fts3_test.c
# Test OBJ files
TESTOBJ = \
test_async.o \
test_autoext.o \
test_backup.o \
test_btree.o \
test_config.o \
test_demovfs.o \
test_devsym.o \
test_func.o \
test_hexio.o \
test_init.o \
test_intarray.o \
test_journal.o \
test_malloc.o \
test_mutex.o \
test_onefile.o \
test_osinst.o \
test_pcache.o \
test_schema.o \
test_server.o \
test_stat.o \
test_tclvar.o \
test_thread.o \
test_vfs.o \
test_wsd.o \
test1.o \
test2.o \
test3.o \
test4.o \
test5.o \
test6.o \
test7.o \
test8.o \
test9.o \
test_fuzzer.o \
test_multiplex.o \
test_quota.o \
test_rtree.o \
test_superlock.o \
test_syscall.o \
test_wholenumber.o \
test_loadext.o \
test_spellfix.o \
test_vfstrace.o \
fts3_term.o \
fts3_test.o
# Header files used by all library source files.
HDR = \
$(TOP)/sqlite3.h \
$(TOP)/btree.h \
$(TOP)/hash.h \
$(TOP)/hwtime.h \
$(TOP)/mutex.h \
$(TOP)/os.h \
$(TOP)/os_common.h \
$(TOP)/pager.h \
$(TOP)/pcache.h \
$(TOP)/sqlite3ext.h \
$(TOP)/sqliteLimit.h \
$(TOP)/sqliteInt.h \
$(TOP)/test_intarray.h \
$(TOP)/test_multiplex.h \
$(TOP)/test_quota.h \
$(TOP)/wal.h \
$(TOP)/vdbe.h \
$(TOP)/tcl.h \
$(TOP)/tclDecls.h \
$(TOP)/tclPlatDecls.h \
$(TOP)/tclTomMathDecls.h \
$(TOP)/tclTomMath.h \
$(TOP)/fts3Int.h
testfixture$(TEXE): $(CC) $(CFLAGS) -o $(SRCOBJ) $(TESTOBJ) $(HDR) -lm $(LIB)
#depend:
# makedepend -Y $(HDR)
clean:
rm -f $(TESTOBJ) $(SRCOBJ)
You should change
testfixture$(TEXE): $(CC) $(CFLAGS) -o $(SRCOBJ) $(TESTOBJ) $(HDR) -lm $(LIB)
to
testfixture$(TEXE): $(TESTOBJ)
$(CC) $(CFLAGS) -o $(SRCOBJ) $(TESTOBJ) $(HDR) -lm $(LIB)
because $(CC) is the command to run, not the target dependencies.

Resources