I am looking for a vba library for solving nonlinear least-squares problems. Usually I use Matlab functions lsqcurvefit(fun,x0,xdata,ydata) and lsqnonlin(fun,x0) which have the same nls algorithm but different problem specification. Does anyone know vba library (function) with problem specification as in lsqnonlin(fun,x0)? Of course it is possible to adjust ALGLIB library but it requires time...
Thanks in advance.
This question will probably get a better response on SuperUser, but here's a possible solution:
In Excel, if you create a Scatter chart of your data, you can right-click on the line in the chart, and choose 'Add Trendline'. This will give you several options for the type ofcurve-fitting you want to do.
However, all the built-in methods are for 'linear' data and will most likely not be optimal for non-linear data.
Related
Excel solver is amazing and wondering if its possible to incorporate the Solver functionality into a web based app instead of keeping everything in Excel. My guess is no because solver solutions are cell based.
On second thought, there is probably a similar solver available with python.
It is possible, but extremely far fetched. You would have to set up Excel on the web server, which makes crazy things to the licensing. Once there, you will have to run Excel in a separate process, but control it from your web app. Using interop in .net might be the least stupid way of doing this.
So just a couple of links to point you in the right direction:
https://www.solver.com/frontline-systems-releases-next-generation-solver-add-in-optimization-excel-online-and-excel-2016
The last few paragraphs point out that you can use the solver with RASOn, JSON C++, R, Python etc in your own apps so that is an option for you.
https://www.solver.com/
Is the homepage of the company that provides the Solver, and access to the more powerful version that exceeds the built-in version in Excel - good though that is. I have not needed the larger version - just turned a model or two around to reduce the computing needs.
I need to solve a linear program (LP) with a mix of binary and continuous variables (MILP).
The data I use as constraints and objectives come from pandas data frame manipulations, so they are in matrix (or I should probably say numpy array) format, and the variables to solve for are sometimes 100's / 1000's.
I know how to use matrices and vectors to setup and solve MILP's in R, but as I coded part of the data handling in python, I wanted to include the LP part in the python script.
So I looked for MILP solvers for python, and I found this post:
Python Mixed Integer Linear Programming
and this one:
https://realpython.com/linear-programming-python/#what-is-mixed-integer-linear-programming
Apart from many links no longer working in the first post, my impression, after browsing through the docs of some of the suggested solvers, was that the LP functions in scipy, which take matrices and vectors as input, cannot solve MILP's (see also this post); on the other hand, interfaces like pulp etc., which do solve MILP's, seem to have their own very special input where one has to specify one by one the equations and name each variable. That would be very inconvenient, as I already have a simple numpy array where each column is a variable and each row is a vector of constraint coefficients.
Does anyone know of a python interface to a MILP solver that takes matrices and vectors as input, instead of requiring to write everything as explicit text equations?
Thanks!
PS I should probably mention that I am looking for interfaces to non-commercial solvers.
EDIT adding my current conclusion, after following the advice from users, for future record and in case it helps someone else
In short: as pulp does the job and comes with a (by all accounts) very good LP solver (CBC), I gave in and accepted to rewrite my problem as a set of equations and inequalities, instead of using the matrix format.
It seems to work, it's indeed fast, at least for the problems I tried so far.
My advice to anyone trying to use it on large problems with many variables would be: look up LpAffineExpression, it's very useful to put together complicated constraints. But first you need to define your variables using LpVariable. A variable can also be a list of variables (that's what stumped me initially).
Before trying pulp, I tried cvxopt, which is easy to install, but then it's true, like the users below said, it has its own bizarre format, and GLPK unfortunately is not a very fast solver.
When I still tried to stick to my matrix input format, I tried cvxpy. On paper this is very good, and in fact would allow me to write the problem as a sum of squares min, which is in fact what my original problem was (I turned it into a LP by replacing sum of squares with sum of abs values, and converting that to LP by known techniques).
I got all the way to writing my objective and constraints in matrix format, and then the solver failed... in fact even the example problem published in the cvxpy documentation failed with the same error. Big waste of time.
Anyway, lesson learned: no point fighting: sometimes you just have to do what python wants you to do ;)
There are some specifics missing and no real definition for how close to exactly what you have as input must be accepted, but Google's OR tools mention a method to define problems for the various solvers they wrap through arrays. This may be workable for you.
For 100s/1000s of variables, this is probably not an option, but I have added an answer to the scipy.optimize question with a toy implementation of branch-and-bound on top of linprog which takes it's args in arrays. If your problem is highly constrained, it might give some answers in a day or two :).
For future reference:
Make sure you have at least pip install scipy>=1.9.3 and use from scipy.optimize import milp as explained here using the HiGHS solvers via method='highs'
I've created a specific model on excel which uses historical returns of firms and aims to find the optimal point between the return and a stop-loss point. Basically I need the optimal stop-loss %. I've been using excel's solver, however, it only finds the local optimum. Can solver somehow find the global optimal point or even a few optimal points? If not, are there any programs that can be recommended to use?
P.S. I am not extremely good at mathematics, and transporting all of the information from excel to an equation or something is probably not an option.
Please help, thanks.
You can either upgrade for the commercial solver which has better support to find a global optimum. Alternatively, you can write a macro that will initialize the various variables to random values and repeatedly launch the optimizer and checking within each loop if a better solution has been found and store the value of the variables before continuing the search.
See https://www.solver.com/upgrade-excel-solver
Note: I am not affiliated with this company. I would rather have you use open source solutions. This is possible with some training.
I have an excel file, with different worksheets and in this file everything is connected and I also have some computations for almost all values (formulas) and I want to do a multi-objective optimization to this work. Since matlab solves multi-objective optimization problems I would like to use it. So I want to know if it is possible to use matlab optimization toolbox as and add-in for excel, so I won't have to do everything again on matlab.
Thanks.
Excel Link most likely isn't going to be of any use here. It's good when you want to use Excel as a front-end, and then push/pull data to/from MATLAB, and for calling MATLAB functions from Excel. But it isn't going to allow you to use the spreadsheet as a cost function within a call to an optimization function, with the cells updating as the optimization progresses (which is what it sounds as if you are trying to do).
You could try setting up an optimization in MATLAB where the cost function that you write uses actxserver to manipulate your spreadsheet. However, that's not going to be trivial to achieve, and you may find that the speed (or lack thereof) of accessing the spreadsheet is prohibitive.
You'll most likely find that the above is harder than rewriting the cost function in m-code (which you're obviously trying to avoid).
Alternatively if you really can't move out of Excel then you should look at using an Excel optimization package. Solver in the first instance (although it is very light weight), or one of the many other optimization add-ins available.
I am new using this Solver function. Is there is way to use Solver for more than 200 sets of data. Maybe using VBA? Really hope someone could help me
No, you need to buy their commercial product (from Frontline Systems) or build one yourself. For Linear/Non-linear optimization there are a lot of open source libraries available in C++ so you may compile a DLL and invoke it from a VBA wrapper.
Also, it's not difficult to implement it completely in VBA. The original guys who developed the solver have given a very very detailed explanation of its workings here. These are the guys who actually made it so you can easily replicate it - especially for the linear case.
A bit dated, but will give you an idea on how to proceed.
https://www.utexas.edu/courses/lasdon/design.htm
Especially useful are the sections where they describe how they build the Jacobian matrix numerically (for Generalized Gradient Descent in NLopt) and how they use a small perturbation to determine the linear coefficients for the Simplex Method. Thus for a linear problem, the only trick is to generate the Coefficient Matrix and then the solver can proceed completely in-memory and generate
the optimal solution
Shadow prices, reduced costs, etc.