New Forum | Previous | Next | (P-PDF) Developers
Topic: Inconsistant math
Conf: (P-PDF) Developers, Msg: 138721
Date: 8/31/2005 06:48 AM
Don't ask why its doing it this way, but this is how datalogics gets the standard coords (96dpi) from literal positions:
I = ((((PageSize.top + 0x00ffff) >> 16) * Scale) >> 16) - ((CosIntegerValue(CosArrayGet(Rect, 3))) * (96/72));
I don't know about you, but to me that math is not only crazy, its horribly inacurate. Because they are using bitshift, you have to use non-decimal (Integer) results. BUT they are using 96/72, which results in 1.33333333333... Obviously using this in an integer math will produce slightly-off results -- and it does.
I tried changing from 96 (dpi) to 150, and calculated the ration between the two -- that ratio is 0.64. Then I took a value (the left of an object) calculated at both DPIs, and got the ratio -- as I suspected, the ratio is 0.665849(etc).. Obviously there is a percision problem here..
Is there a better way to get the screen coordiates than this?