Sorry this may seem like a stupid question, I'm quite new to the Netlify platform but I am using the drag and drop feature to update my static website. After dragging and dropping the Deploys page simply has a Production uploading and there is nothing in the deploy log.
Is there a hard limit on what I am uploading (I haven't uploaded anything over 2GB) or do I have to upload via the Command Line Interface?
Thank you
disclaimer: I work for Netlify.
Drag and drop deploys to Netlify are the least reliable type. They rely on a bunch of things working perfectly:
your computer - should have quite a bit of memory since the entire site needs to fit in memory potentially twice (if you use a zipfile)
your network connection - needs to be ROCK SOLID. if the connection drops during deployment, it is unlikely to resume
your site size. 2GB isn't "a limit" but I wouldn't expect a drag and drop deploy of over 100MByte to work well on a regular basis.
CLI or Git-backed deployments are far more reliable, plus they give us some opportunity to debug what went wrong which drag and drop deploys do not (no logging on our side, no debug logs possible on your side).
If the deployment hung in uploading status for more than a couple of minutes, it will never complete and you should try again (preferably via the CLI :)).
In general questions like these are probably better handled in Netlify's helpdesk. In this case I would have given you the same answer there.
Related
I have a file which is around 1.2 GB size and I want to call an instance to it while formulating results for my website. Is it possible to make the instance the same for all users of the website? According to my understanding, for eg Heroku, as all create separate instances of the website for every user, is there any way to make it happen. I apologize in advance if the question is naive !!
Bummer, heroku only allows you to have 500mb of files in your deploys.
If not for that, you could just commit that big file to your repo and use it on your site.
Heroku doesn't create an instance of the app for every user but for every deploy. Once you deploy your application, your entire old server is thrown away and a new one is created.
Let's say you need a temporary file, you could put it on /tmp/something and then use it for a while. Once you make a deploy, as the server is discarded and a new one is spawn, that file wouldn't be there and you'd have to recreate it for the first request.
Anyway, that doesn't look good. Even if heroku would let you store the file there, you'd also have to parse and dig through it in order to display your results for your site, which is likely to make you run out of memory too.
I would recommend you to review your approach to this problem, maybe break that file in small pieces or store it in a database to perform smaller calculations.
If there is absolutely no way around it, you could have a server in some other server like digital ocean and build a small api to perform the calculations there and make your site call it.
If you want, I can give you a hand on switching the approach for it, just post a different question, comment the link to it and I'll give it a shot.
I'm moving my Three.js app and its customized node.js environment, which I've been running on my local machine to Google Cloud. I want to test things out there, and hopefully soon get some early alpha testing going with other people.
I'm not sure which is the wiser way to go... to upload the repo I've been running locally as-is onto a VM which users would then access via the VM's external IP until I get a good name to call this app... or merge my local node.js environment with what's available via the Google App Engine and run it on GAE.
Issues I'm running into with the linux VM approach... I'm not sure how to do the equivalent on the VM of what I've been doing locally. In Windows Powershell I cd into the app directory and then enter node index.js. I'm assuming by this method of deployment that I can get the app running as soon as the browser hits the external IP. I should mention too that the app will allow users to save content as well as upload images, and eventually, 3D models as well as json datasets.
Issues I'm running into with the App Engine approach: it looks like I only have access to a linux-based command line, and have to install all the node.js modules manually. Meanwhile I have a bunch of files to upload, both the server-side node files and all the frontend stuff. I don't see where to upload those files, and ultimately what I'd like to do is have access to a visual, editable file-tree interface, as I have in Windows and FileZilla, so I can swap files in and out, etc. Alternatively I suppose I could import a repo from Github? Github would be fine as long as I can visually see what's happening. Is there a visual interface for file structure available in GAE somewhere? Am I missing something?
I went through the GAE "Hello World" tutorial and that worked fine, but was left scratching my head afterward regarding how to actually see and edit the guts of the tutorial app, or even where to look for the files.
So first off, I want to determine what's the better approach, and then if possible, determine how to make the experience of getting my app up there and running a more visual, user-friendly experience.
Thanks.
There are many things to consider when choosing how to run an app, but my instinct for your use case is to simply use a VM on GCE. The most compelling reason for this is that it's the most similar thing to what you have now. You can SSH into the machine and run nohup node index.js & (or node index.js inside tmux/screen if you prefer) and it will start the app and not stop it when you log out of SSH. You can use SCP / SFTP with whatever GUI client you want to upload files. You don't have to learn anything new! If you wanted to, you could even use a Windows VM (although I think you have to pay a little more than for a comparable Linux VM due to the licensing fees).
That said, the other way is arguably more "correct" by modern development standards, but it will involve a lot more learning that will prevent you from getting your app running somewhere other than your laptop in the short term:
First, you'll need to learn about Docker and stateless containers, which is basically what your app runs inside of on AppEngine.
Next, you'll need to learn how to hook up a separate stateful service (database, file server, ...) to your app's container so you can store your files, etc. in it, and then probably rewrite your app somewhat to use it to store stuff.
Next, you'll probably want some way to automatically deploy this from code instead of manually doing it, which gets you into build systems, package managers, artifact storage, continuous integration systems, and on and on and on.
This latter path is certainly what you should choose for a long-running production service if you work with a big team of developers -- but that doesn't mean that it's necessarily the right path for your project today. If you don't care about scaling up automatically, load balancing between nodes, redundant copies of your app running in different regions in case there's a natural disaster, etc., then go with the easy way for now, and you can learn new ways to improve the service when they're actually needed.
I'm currently running Umbraco on a web app for Microsoft Azure. Anytime I enable scaling out and the web app starts scaling out, I get the error:
"Process cannot access the file, Examine Indexes write.lock because it is being used by another file.
The website then needs to be restarted before it becomes fully functioning again. Is there a setting on Umbraco that I'm missing?
Or is it something that happens with Azure Web Apps Auto Scaling features?
This sounds like an issue with the indexes. Your index appears to be getting locked when scaling out. Ideally if you're running on a load balanced environment, you should have a single index for all environments instead of on a per instance basis. I've used Azure Search in the past and it's worked perfectly, swapping out the index isn't too difficult with Umbraco, plenty of information available online. Good example here
In the future you shouldn't need to restart the entire site, rebuilding the indexes should be fine.
Also, what version of Umbraco are you running? This may be of some help, I encountered some similar issues a few months ago - unrelated to scaling though.
https://issues.umbraco.org/issue/U4-10735
Sounds like you need to isolate your index files so they aren’t shared across the difference instances and don’t lock each other out. There’s a few ways to do this based on the version you are running, but in 7.3, i think you update the index file location to include the instance name like ~/App_Data/TEMP/ExamineIndexes/{machinename}/Internal/
For more details, see https://our.umbraco.com/documentation/getting-started/setup/server-setup/load-balancing/flexible#if-you-plan-on-using-auto-scaling
I am currently changing our database deployment strategy to use FluentMigration and have been reading up on how to run this. Some people have suggested that it can be run from Application_Start, I like this idea but other people are saying no but without specifying reasons so my questions are:
Is it a bad idea to run the database migration on application start and if so why?
We are planning on moving our sites to deploying to azure cloud services and if we don't run the migration from application_start how should/when should we run it considering we want to make the deployment as simple as possible.
Where ever it is run how do we ensure it is running only once as we will have a website and multiple worker roles as well (although we could just ensure the migration code is only called from the website but in the future we may increase to 2 or more instances, will that mean that it could run more than once?)
I would appreciate any insight on how others handle the migration of the database during deployment, particularly from the perspective of deployments to azure cloud services.
EDIT:
Looking at the comment below I can see the potential problems of running during application_start, perhaps the issue is I am trying to solve a problem with the wrong tool, if FluentMigrator isn't the way to go and it may not be in our case as we have a large number of stored procedures, views, etc. so as part of the migration I was going to have to use SQL scripts to keep them at the right version and migrating down I don't think would be possible.
What I liked about the idea of running during Application_Start was that I could build a single deployment package for Azure and just upload it to staging and the database migration would be run and that would be it, rather thank running manual scripts, and then just swap into production.
Running migrations during Application_Start can be a viable approach. Especially during development.
However there are some potential problems:
Application_Start will take longer and FluentMigrator will be run every time the App Pool is recycled. Depending on your IIS configuration this could be several times a day.
if you do this in production, users might be affected i.e. trying to access a table while it is being changed will result in an error.
DBA's don't usually approve.
What happens if the migrations fail on startup? Is your site down then?
My opinion ->
For a site with a decent amount of traffic I would prefer to have a build script and more control over when I change the database schema. For a hobby (or small non-critical project) this approach would be fine.
An alternative approach that I've used in the past is to make your migrations non-breaking - that is you write your migrations in such a way they can be deployed before any code changes and work with the existing code. This way both code and migrations both can be deployed independently 95% of the time. For example instead of changing an existing stored procedure you create a new one or if you want to rename a table column you add a new one.
The benefits of this are:
Your database changes can be applied before any code changes. You're then free to roll back any breaking code changes or breaking migrations.
Breaking migrations won't take the existing site down.
DBAs can run the migrations independently.
I have been playing around with Kudu on an IIS development server and succeeded in making it deploy a hello world site etc.
But I was wondering if there was some resources on how to deploy Kudu in larger environments. So that you can quickly add new server nodes (virtual) to balance load.
Is there some approach to manage multiple Kudu deployments from a centralized location? I know It has an REST API and obviously we can probably use this, but that sort of requires some development, so was just looking to see if there was an existing solution out there.
But So far I have either searched after the wrong thing on Google, or there isn't much of what I need.
Does anyone have any experience in running it in larger environments with many servers?