we have log files generating from different sources and storing in directories & sub-directories. The directory structure is something like below.
I want to tar & zip the file older than 30 days maintaining the directory structure and delete the archived files after the tar & zip.
Can someone help me with this? How this can be achieved.
dataload
├── apiConnectorApp
│ ├── csv
│ │ ├── 20210216231308
│ │ ├── batch1
│ │ ├── batch2
│ │ └── batch3
│ ├── day1Load
│ ├── logs
│ └── sql
├── configs
│ ├── eSite
│ │ ├── CKB2B
│ │ │ ├── CatalogEntryAssociations
│ │ │ ├── CatalogGroup
│ │ │ └── CatalogGroupCatalogEntryRelation
│ │ └── THB2B
│ │ ├── CatalogEntryAssociations
│ │ ├── CatalogGroup
│ │ └── CatalogGroupCatalogEntryRelation
find dataload -type f -mtime +30 -exec tar -rf archive.tar {} + -exec rm -f {} +
Run find with dataload as the parent directory. Process files that have been modified more that 30 days ago. Append as many files as possible (+) to an archive file "archive.tar" by using the tar -r flag. Then remove the files.
Related
I have a directory with subdirectories. I want to unzip every .zip file in this directory structure recursively and locally. (Unzip next to the .zip file)
For example:
.
├── A
│ ├── A2
│ │ └── content1.zip
│ ├── content2.zip
│ └── content3.zip
├── B
│ └── content4.zip
└── content5.zip
After unzip:
.
├── A
│ ├── A2
│ │ ├── content1.txt
│ │ └── content1.zip
│ ├── content2.txt
│ ├── content2.zip
│ ├── content3.txt
│ └── content3.zip
├── B
│ ├── content4.txt
│ └── content4.zip
├── content5.txt
└── content5.zip
I prefer if it's working both on linux and windows.
With python3
import os
import sys
import zipfile
for dir in os.walk(sys.argv[1]):
#dir[0] - current directory
#dir[1] - directories in current directory
#dir[2] - files in current directory
for file in dir[2]:
if file.endswith(".zip"):
with zipfile.ZipFile(dir[0] + "/" + file, 'r') as zip_ref:
zip_ref.extractall(dir[0])
Run in current directory:
python3 unzip.py .
My current directory contents are:
$ tree
├── README.md
├── deploy.sh
├── grizzly
│ ├── configs
│ │ ├── nginx-conf.yml
│ │ └── proxy-conf.yml
│ ├── deployments
│ │ ├── api.yml
│ │ ├── celery.yml
│ │ └── proxy.yml
│ ├── secrets
│ └── services
│ ├── api.yml
│ └── proxy.yml
├── ingress.yml
└── shared
├── configs
│ └── rabbitmq.yml
└── env
└── variables.yml
I plan to create a script that will run $ kubectl apply for all files in this tree.
My thought is to get all the child directories then just have all those child directories(expected to have the yml files) execute $ kubectl apply for my resources to be created.
This is an instance of the XY Problem. You want to apply all yamls which are somewhere within the directory structure of the current directory.
Just run:
kubectl apply -f . --recursive
If you want to filter the files based on certain conditions you can use a construct like
find . -type f | grep 'api.yml' | xargs -n 1 kubectl apply -f
I'm following Swaroop's Byte of Vim, and have reached the chapter on personal information management, where it says to install the Viki plugin. The instructions are as follows (and I have no real idea of what is going on, but):
Download multvals.vim [2] and store as $vimfiles/plugin/multvals.vim
Download genutils.zip [3] and unzip this file to $vimfiles
Download Viki.zip [4] and unzip this file to $vimfiles (make sure all the folders and files under the 'Viki' folder name are stored directly in the $vimfiles folder)
After this I open a new text file in vim and run the commmand
:set filetype=viki
but I get a whole slew of errors. I've tried clearing out my ~/.vim folder and reinstalling everything, along with tlib this time as specified on the Viki vimscript page, and extracted the version 4.0 viki.vba instead of using the version 4.08 zip file, but I'm still getting errors about non-existent functions:
Error detected while processing home/user/.vim/ftplugin/viki.vim:
line 100
E117: Unknown function: tlib#balloon#Register
I don't really know what's going on, and am quite a new Vim user, so please be patient. Right now my ~/.vim directory tree looks like this:
.vim
├── autoload
│ ├── genutils.vim
│ ├── tlib
│ │ ├── eval.vim
│ │ ├── list.vim
│ │ ├── notify.vim
│ │ ├── persistent.vim
│ │ ├── progressbar.vim
│ │ ├── TestChild.vim
│ │ └── vim.vim
│ ├── tlib.vim
│ ├── viki
│ │ ├── enc_latin1.vim
│ │ └── enc_utf-8.vim
│ ├── viki_anyword.vim
│ ├── viki_latex.vim
│ ├── viki_viki.vim
│ └── viki.vim
├── colors
│ └── molokai.vim
├── compiler
│ └── deplate.vim
├── doc
│ ├── tlib.txt
│ └── viki.txt
├── ftplugin
│ ├── bib
│ │ └── viki.vim
│ └── viki.vim
├── plugin
│ ├── 02tlib.vim
│ ├── genutils.vim
│ ├── multvals.vim
│ └── viki.vim
└── test
└── tlib.vim
Any help is much appreciated.
The info is outdated. You need to install a current versions of tlib and viki from:
https://github.com/tomtom/viki_vim
https://github.com/tomtom/tlib_vim
I tried to put ignored setting in .vimrc
But when I used the ctrlp to search under rails app folder
It still search the vendor folder, so it took lots of time.
But when the search was done, I couldn't search anything under the vendor
It was so strange! How to fix it.
Here is my .vimrc setting file.
http://d.pr/i/yMtK
http://d.pr/i/Hy4u
" Sane Ignore For ctrlp
let g:ctrlp_custom_ignore = {
\ 'dir': '\.git$|vendor\|\.hg$\|\.svn$\|\.yardoc\|public\/images\|public\/system\|data\|log\|tmp$',
\ 'file': '\.exe$\|\.so$\|\.dat$'
\ }
When I appended the code in the end of .vimrc
217 let g:NERDTreeIgnore=['\~$', 'vendor']
218 set wildignore+=*\\vendor\\**
It worked when my first time to use the CTRLP to search under the RAILS app folder,
But still NOT worked in the following times.
I guess maybe there are some settings will disable the ignored setting ？
Here are the structure of my folder
.
├── Gemfile
├── Gemfile.lock
├── README.rdoc
├── Rakefile
├── app
│ ├── assets
│ ├── controllers
│ ├── helpers
│ ├── mailers
│ ├── models
│ ├── uploaders
│ ├── views
│ └── workers
├── auto.sh
├── config
│ ├── application.rb
│ ├── application.yml
│ ├── boot.rb
│ ├── database.yml
│ ├── environment.rb
│ ├── environments
│ ├── initializers
│ ├── locales
│ ├── macbookair_whenever_schedule.rb
│ ├── menu_navigation.rb
│ ├── navigation.rb
│ ├── resque.god
│ ├── resque_schedule.yml
│ ├── routes.rb
│ ├── schedule.rb -> ubuntu_whenever_schedule.rb
│ ├── tinymce.yml
│ └── ubuntu_whenever_schedule.rb
├── config.ru
├── db
│ ├── development.sqlite3
│ ├── migrate
│ ├── migrate_should_be_skip
│ ├── production.sqlite3
│ ├── schema.rb
│ └── seeds.rb
├── doc
│ └── README_FOR_APP
├── lib
│ ├── assets
│ ├── auto_tools
│ ├── tasks
│ └── url_automation_module.rb
├── log
│ ├── apalog
│ ├── development.log
│ ├── passenger.80.log
│ ├── production.log
│ └── prodution.log
├── output_name
├── public
│ ├── 404.html
│ ├── 422.html
│ ├── 500.html
│ ├── exports
│ ├── favicon.ico
│ ├── results.zip
│ ├── robots.txt
│ ├── sandbox
│ └── uploads
├── script
│ ├── delayed_job
│ └── rails
├── test
│ ├── fixtures
│ ├── functional
│ ├── integration
│ ├── performance
│ ├── test_helper.rb
│ └── unit
├── test.sh
├── tmp
│ ├── cache
│ ├── pids
│ ├── restart.txt
│ ├── sessions
│ └── sockets
├── tmplog
└── vendor
└── bundle
If you type :help ctrlp-options and read a bit, you will find:
Note #1: by default, wildignore and g:ctrlp_custom_ignore only
apply when globpath() is used to scan for files, thus these options
do not apply when a command defined with g:ctrlp_user_command is
being used.
Thus, you may need to unlet g:ctrlp_user_command (possibly set to a default command) to actually use wildignore as advised by #TomCammann. For instance, in your ~/.vimrc, add:
if exists("g:ctrlp_user_command")
unlet g:ctrlp_user_command
endif
set wildignore+=*\\vendor\\**
After that, you need to refresh your ctrlp cache: in Vim, press F5 in ctrlp mode, or run :CtrlPClearAllCaches, or remove the cache directory directly in your shell:
rm -r ~/.cache/ctrlp/ # On Linux
part of my .vimrc file . perhaps it will help
set wildignore+=*/.git/*,*/.hg/*,*/.svn/*,*/.idea/*,*/.DS_Store,*/vendor
You can use the wildignore vim setting which CtrlP will pick up on.
set wildignore+=*\\vendor\\**
Check if you are using some specific search command, like:
let g:ctrlp_user_command = 'find %s -type f' " MacOSX/Linux
let g:ctrlp_user_command = 'dir %s /-n /b /s /a-d' " Windows
This kind of configuration ignores the g:ctrlp_custom_ignore option.
wildignore may be used by other commands, the reason of failure for g:ctrlp_custom_ignore is g:ctrlp_user_command, for example, here is mine:
if executable('rg')
let g:ctrlp_user_command = 'rg %s --files --hidden --color=never --glob ""'
endif
For this case, rg has own ignore way, just put .git to .gitignore, rg will not search any files in .gitignore.
I have a directory-tree with a lot of files in it. I'd like to copy all of those files into one new directory, but with all files located in the base of the folder.
So I have something like this:
images
├── avatar.png
├── bg.jpg
├── checkbox.png
├── cross.png
├── datatables
│ ├── back_disabled.png
│ ├── back_enabled.png
│ ├── forward_disabled.png
│ ├── forward_enabled.png
│ ├── sort_asc.png
│ ├── sort_asc_disabled.png
│ ├── sort_both.png
│ ├── sort_desc.png
│ └── sort_desc_disabled.png
├── exclamation.png
├── forms
│ ├── btn_left.gif
│ ├── btn_right.gif
│ ├── checkbox.gif
│ ├── input
│ │ ├── input_left-focus.gif
│ │ ├── input_left-hover.gif
│ │ ├── input_left.gif
│ │ ├── input_right-focus.gif
│ │ ├── input_right-hover.gif
│ │ ├── input_right.gif
│ │ ├── input_text_left.gif
│ │ └── input_text_right.gif
│ ├── radio.gif
│ ├── select_left.gif
│ ├── select_right.gif
And I'd like something like this:
new_folder
├── avatar.png
├── bg.jpg
├── checkbox.png
├── cross.png
├── back_disabled.png
├── back_enabled.png
├── forward_disabled.png
├── forward_enabled.png
├── sort_asc.png
├── sort_asc_disabled.png
├── sort_both.png
├── sort_desc.png
├── sort_desc_disabled.png
├── exclamation.png
├── btn_left.gif
├── btn_right.gif
├── checkbox.gif
├── input_left-focus.gif
├── input_left-hover.gif
├── input_left.gif
├── input_right-focus.gif
├── input_right-hover.gif
├── input_right.gif
├── input_text_left.gif
├── input_text_right.gif
├── radio.gif
├── select_left.gif
├── select_right.gif
I'm pretty sure there is a bashcommand for that, but I haven't found it yet. Do you have any ideas?
CS
find /source-tree -type f -exec cp {} /target-dir \;
you are looking for ways to flatten the directory
find /images -iname '*.jpg' -exec cp --target-directory /newfolder/ {} \;
find all files iname in case insensitive name mode.
cp copy once to --target-directory named /newfolder/.
{} expand the list from find into the form of /dir/file.jpg /dir/dir2/bla.jpg.
On zsh:
cp /source/**/* /destination
$ cd images && cp ** new_folder/