I have a server deployed on Heroku through Heroku auto deployment method from GitHub. On that server I have a file named subscription.json which contains user data whenever user is registered. I want to see that file.
How can I access that file?
If that file is in your repository you should be able to read it like any regular file.
However, this isn't going to work on Heroku:
which contains user data whenever user is registered
Heroku's filesystem is dyno-local and ephemeral. Any changes you make to it will be lost the next time your dyno restarts. This happens frequently (at least once per day).
For saving things like user uploads, Heroku recommends using a third-party service like Amazon S3. But in this case I think a database would be a much more appropriate solution.
Heroku has its own PostgreSQL service that is very well supported and available out of the box. If you prefer to use another database there are plenty of options.
Related
I've googled this question a lot, but I haven't found right answer.
I've built an example app on NodeJS without Database connectivity. While developing I stored data in separate dir - "fakeDB", which contains "tests", "questions", "users" dirs and so on. In "tests" there are JSON files represent test data (a set of questions and answers).
When I deployed app on Heroku, tests stored correctly. When new test created, it is saved in "tests" dir and I have an access to it later.
But when I push a new commit to GH repo, tests that were created in Heroku, have been deleted.
How can I get copy of my Heroku repo on local machine?
NOTE: I've run heroku run bash and on ls it printed the list of local files, not from remote. Also, I've run git pull heroku to separate dir, but there were also a set of my previous files without created on Heroku.
Heroku's filesystem is ephemeral:
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
This means that you can't reliably create files and store them on the local filesystem. They aren't shared across dynos, and they periodically disappear. Furthermore, Heroku doesn't provide a mechanism for easily retrieving generated files.
The official recommendation is to use something like Amazon S3 for storing uploads, generated files, etc. Of course, depending on what's in your files a database might be a better fit.
I am trying to avoid using a DB in my simple RESTful app.
I created a "posts.txt" file which has all the posts in it, the app reads from this file and creates the posts array (JSON.parse).
The problem is, when I "git push heroku master" it, the "posts.txt" in heroku gets overriden and thus I lose all the posts created by guests.
I tried to .gitignor this file but it seems I just do it worng (or that I didn't understand the idea of "untracking" a file).
What can I do in order to prevent the overriding (I just don't want to push a new "posts.txt" into heroku every time)?
Due to your Heroku app potentially being run on multiple servers over time, there is no way to ensure that your posts.txt file will remain consistent overtime. Also as you make changes, and as you have noted, it can easily get overwritten.
Heroku can terminate your application and start it on another server as needed. Almost like a server-less type setup.
That means there is no real way to ensure a stable data persistence on Heroku without some type of database layer.
Great point mentioned in the comments that I forgot to mention. The file will also be deleted after cycling because the filesystem is ephemeral. You can find more information about file uploads missing/deleted on Heroku's site here.
One other thing about this is even you are using some type of VPS or something like that, you'd still run into the problem of how to sync the posts down to your local machine during development and ensuring that stays in sync. Database layer is for sure the way to go here.
I have a node.js app on heroku and I sometimes need to write files to heroku. Do you know how to see those files? Should I delete them after I am finished using them? I do not want to use memory for no reason.
Heroku (and other container based platforms) are different from traditional servers that you might be used to. It's worth bearing in mind that the Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy.
If you really needed to check a file on a running dyno (let's say to debug an issue with a file upload) it is possible to login using Heroku Exec https://devcenter.heroku.com/articles/exec
That said, you really shouldn't be using the filesystem for anything other than temporary files. Instead you should aim to use external services for persistent storage as described here: https://12factor.net/
For example, if you are handling file uploads you could try storing these on a service like Amazon S3.
I made a discordJs bot which saves data in a file.
Everything is hosted on Heroku and all works good.
But everyday, Heroku reset my file.
Why can't i keep my files everyday ?
Here's the full explanation from Heroku docs:
The Heroku filesystem is ephemeral - that means that any changes to
the filesystem whilst the dyno is running only last until that dyno is
shut down or restarted. Each dyno boots with a clean copy of the
filesystem from the most recent deploy. This is similar to how many
container based systems, such as Docker, operate.
In addition, under normal operations dynos will restart every day in a
process known as "Cycling".
These two facts mean that the filesystem on Heroku is not suitable for
persistent storage of data. In cases where you need to store data we
recommend using a database addon such as Postgres (for data) or a
dedicated file storage service such as AWS S3 (for static files). If
you don't want to set up an account with AWS to create an S3 bucket we
also have addons here that handle storage and processing of static
assets https://elements.heroku.com/addons
Source: https://help.heroku.com/K1PPS2WM/why-are-my-file-uploads-missing-deleted
I am a total beginner in web development using MEAN Stack. I am considering to deploy my app on Heroku for this particular project. I am trying to create an application where the user would upload large excel files(>200MB). The application needs to process the excel file and then export the resultant file of similar size to another service. It is very hard for me to figure out the following:
Would it be possible to save a file on disk in Heroku?
For this process do I need to save the file somewhere on server at all, or would it be possible to just process the file keeping in memory and then export?
Would it be possible to process large files from disk or should I try to save the information in a database?
Are there any Heroku constraints that limit the size of file to be processed within memory?
The information I have looked up seems too involved to be understood by a beginner like me. I haven't had much success with what I have tried and I don't want to spend too much time trying all these possibilities.
How should I proceed?
Thanks.
You should not attempt to save your files on the local file system of a Heroku dyno. Your heroku dynos can recycle without warning at any time, and anything you "stored" on the local file system of your dyno before the recycle will disappear.
Instead, you will probably need some kind of external storage for your files, such as Amazon S3.
You could optionally use a service such as Bucketeer to simplify provisioning your Amazon S3 storage from within Heroku.