Tags: anerror, ascoefficients, fminsearch, function, initial, input, matlab, optimize, parameters, programming

Initial parameters to fminsearch function

On Programmer » Matlab

8,481 words with 3 Comments; publish: Wed, 30 Apr 2008 19:59:00 GMT; (20062.50, « »)

Hi,

I've been using fminsearch to optimize 5 parameters for an

error function. Three of the input parameters are used as

coefficients so their initial value is set to 1. The other

2 parameters are more like an offset to other variables so I

use the initial value of 0 so my function call looks like this:

OptArg=optimset('MaxFunEvals',10000,'Max

Iter',10000,'TolX',1e-6,'TolFun',1e-

6);

x0 = [1, 1, 1, 0, 0];

[x,MSE]=fminsearch(.matlab.questionfor.info.pmodel_err,x0,OptArg

);

where pmodel_err is my error function.

If I use a non zero inisial step such as:

x0 = [1, 1, 1, 10, 5];

I get good results, but if using zero initial values, the

final optimized x(4) and x(5) are close to zero. I

monitored the values that are passed to my error function as

the trial for x and with x0(4:5)=0 those values stay close

to zero.

It seems to me that fminsearch is sensitive to its initial

input parameters which is not a good thing for an

optimization tool. Does anybody have similar problem and

how do you go about fixing it?

Thanks,

Arash

All Comments

Leave a comment...

  • 3 Comments
    • "Arash " <asamani.matlab.questionfor.info.gmail.com> wrote in message

      <fl0b0u$cto$1.matlab.questionfor.info.fred.mathworks.com>...

      > Hi,

      > I've been using fminsearch to optimize 5 parameters for an

      > error function. Three of the input parameters are used as

      > coefficients so their initial value is set to 1. The other

      > 2 parameters are more like an offset to other variables so I

      > use the initial value of 0 so my function call looks like this:

      > OptArg=optimset('MaxFunEvals',10000,'Max

      Iter',10000,'TolX',1e-

      6,'TolFun',1e-6);

      > x0 = [1, 1, 1, 0, 0];

      > [x,MSE]=fminsearch(.matlab.questionfor.info.pmodel_err,x0,OptArg

      );

      > where pmodel_err is my error function.

      > If I use a non zero inisial step such as:

      > x0 = [1, 1, 1, 10, 5];

      > I get good results, but if using zero initial values, the

      > final optimized x(4) and x(5) are close to zero. I

      > monitored the values that are passed to my error function as

      > the trial for x and with x0(4:5)=0 those values stay close

      > to zero.

      > It seems to me that fminsearch is sensitive to its initial

      > input parameters which is not a good thing for an

      > optimization tool. Does anybody have similar problem and

      > how do you go about fixing it?

      Virtually all optimizers are dependent on their

      starting values. Some are more robust than other

      others against convergence problems. But if an

      optimizer is placed in the basins of attraction of

      two distinct local minimizers, then you will get

      two different results.

      Nothing that you have said has yet convinced me

      that your objective function is well posed. For

      example, from your meager description, consider

      the model

      y = (a0 + a1)*x

      From data, estimate both a0 and a1 as coefficients

      in the model. Here a1 might be described as an

      "offset" to a0. Can you uniquely estimate both

      a0 and a1?

      If you really do have a validly posed optimization

      problem (hey, it does happen, 8-) then you might

      consider the application of constraints on the

      problem. Surely you know something about this

      system, since you are unhappy with the results

      in at least some of your efforts. So why not apply

      constraints that embody your knowledge of this

      system?

      Fminsearch is a rather basic tool, so you might

      also consider using a different code, perhaps

      one from the optimization toolbox.

      John

      #1; Wed, 30 Apr 2008 20:00:00 GMT
    • "John D'Errico" <woodchips.matlab.questionfor.info.rochester.rr.com> wrote in

      message <fl0dis$4sl$1.matlab.questionfor.info.fred.mathworks.com>...

      > "Arash " <asamani.matlab.questionfor.info.gmail.com> wrote in message

      > <fl0b0u$cto$1.matlab.questionfor.info.fred.mathworks.com>...

      like this:

      OptArg=optimset('MaxFunEvals',10000,'Max

      Iter',10000,'TolX',1e-

      > 6,'TolFun',1e-6);

      > Virtually all optimizers are dependent on their

      > starting values. Some are more robust than other

      > others against convergence problems. But if an

      > optimizer is placed in the basins of attraction of

      > two distinct local minimizers, then you will get

      > two different results.

      > Nothing that you have said has yet convinced me

      > that your objective function is well posed. For

      > example, from your meager description, consider

      > the model

      > y = (a0 + a1)*x

      > From data, estimate both a0 and a1 as coefficients

      > in the model. Here a1 might be described as an

      > "offset" to a0. Can you uniquely estimate both

      > a0 and a1?

      > If you really do have a validly posed optimization

      > problem (hey, it does happen, 8-) then you might

      > consider the application of constraints on the

      > problem. Surely you know something about this

      > system, since you are unhappy with the results

      > in at least some of your efforts. So why not apply

      > constraints that embody your knowledge of this

      > system?

      > Fminsearch is a rather basic tool, so you might

      > also consider using a different code, perhaps

      > one from the optimization toolbox.

      > John

      John,

      Thanks as always for your reply.

      My function a model to generate blood pressure waveform and

      then compare it with the actual measured blood pressure.

      The model is well defined and the fit for the model vs. the

      measured waveforms are good. My function returns the MSE of

      these 2 waveforms. I ended up suing non zero initial values

      but I have to justify why I expect the offset in first

      place; it would be much nicer if I could start offsets from

      zero. If I pick a non zero offset (1 or 10 or 100 -- units

      don't matter), fminsearch converges to the same results, but

      using zero or close to zero initial value takes it somewhere

      else.

      I don't have the optimization toolbox and it's not that easy

      to put constraints on fminsearch. Is there any other

      alternatives outside of optimization toolbox for that?

      Thanks,

      Arash

      #2; Wed, 30 Apr 2008 20:01:00 GMT
    • "Arash " <asamani.matlab.questionfor.info.gmail.com> wrote in message

      <fl0ipg$kbg$1.matlab.questionfor.info.fred.mathworks.com>...

      > My function a model to generate blood pressure waveform and

      > then compare it with the actual measured blood pressure.

      > The model is well defined and the fit for the model vs. the

      > measured waveforms are good. My function returns the MSE of

      > these 2 waveforms. I ended up suing non zero initial values

      > but I have to justify why I expect the offset in first

      > place; it would be much nicer if I could start offsets from

      > zero. If I pick a non zero offset (1 or 10 or 100 -- units

      > don't matter), fminsearch converges to the same results, but

      > using zero or close to zero initial value takes it somewhere

      > else.

      > I don't have the optimization toolbox and it's not that easy

      > to put constraints on fminsearch. Is there any other

      > alternatives outside of optimization toolbox for that?

      But if you start an optimizer in the wrong basin

      of attraction, it will converge to the wrong solution.

      Deliberately giving it poor starting values will not

      produce an unbiased solution, just the wrong one.

      An optimizer is just a tool that tries to walk downhill

      from your starting point. It cares not that the downhill

      direction leads it into the wrong valley.

      There are a few tools on the FEX that will allow you

      to impose constraints. My own fminsearchbnd and

      fminsearchcon are two simple ones, built on top of

      fminsearch. There is also now an fminsearchbnd_new

      that improves on a few things with the newer releases

      of matlab that have come out since I wrote those tools.

      http://www.mathworks.com/matlabcent...ge/loadFile.do?

      objectId=8277&objectType=file

      http://www.mathworks.com/matlabcent...oadCategory.do?

      objectId=5&objectType=Category

      John

      #3; Wed, 30 Apr 2008 20:02:00 GMT