Changed tolerance in lattice reduction

Lattice reduction is used in plumed to convert box to its
"least skewed" form. I changed a bit the threashold since it
was giving problems with high optimization flags.
Additionally, I added some warning when too many iterations
are performed (which indicates a numerical issue)