[Tig] GraphicsMagick & SMPTE DPX

Martin Euredjian ecinema
Mon May 9 19:23:30 BST 2005


> Can you (or anyone else) shed some light as to how big/little endian 
file organization should impact the DPX raster image storage?

The "big/little endian" refers to the way the host CPU stores multi-byte
values in memory.  A 16 bit value could be stored with the most significant
byte first or last, for example.

If I remember correctly, files coming off SGI systems are LSB first.
Windows and Linux (Intel based) systems would require MSB first and,
therefore, byte swapping on read and write (if the files need to go back to
SGI).

When using DPX it is clear that the fastest display/manipulation/playback
solution is one that does not alter the natural order of the CPU under which
it runs.  Doing so would mean that at least three byte (or multi-byte) swap
operations would be required each time you read or write to disk or from/to
any buffer that holds unprocessed DPX data.


Upon loading the file you'd set a flag based on the magic number:

	// If this file was created on a big-endian system, enable
byte-swapping
	m_bEnableByteSwap = (m_MagicNumber == 0x58504453) ? TRUE : FALSE;


You finish loading the file and then:

	// If the MagicNumber comes out swapped, this is an Intel (or?)
processor
	// that stores multi-byte numbers least-significant-byte first.
	// Motorola (and?) processors store the most significant byte first.
		
	if(m_bEnableByteSwap) SwapAllBytes();


You could do it as you read off disk I suppose.  However, this would mean
paying a penalty during disk read time.  I'd rather get the file into memory
as quickly as possible and then deal with endianness.

Having said that, I didn't bother to run tests to see if the byte swapping
can be done during disk read without impacting performance.  There's a
possible argument there given that you might not have to touch the same data
twice and that disk I/O will always be slower than memory I/O, in other
words, maybe you can sneak the byte swapping during gaps in disk I/O data
flow.

Here's C++ code from one of my DPX viewers that shows a low-performance
means to swap bytes.  By overloading the functions C++ takes care of
deciding which function to use.  Of course, there's a performance hit due to
this.  The best solution is to use assembly language and optimize for speed.
The i486 family of processors (Pentium included) have an instruction just
for this purpose...with some problems.  "BSWAP" converts between little/big
endian data in a 32 bit register.  16 bit data is mangled (undefined) and
so, BSWAP can't be used.  The same is true of any floating point format.
So, you still have to roll your own.
 

void CDPXViewDoc::SwapBytes(DWORD &value)
{
	// Flop the bytes on a DWORD to deal with the way different
processors
	// store multi-byte numbers
	char temp;
	temp = ((char*)(&value))[0];
	((char*)(&value))[0] = ((char*)(&value))[3];
	((char*)(&value))[3] = temp;

	temp = ((char*)(&value))[1];
	((char*)(&value))[1] = ((char*)(&value))[2];
	((char*)(&value))[2] = temp;
}

void CDPXViewDoc::SwapBytes(WORD &value)
{
	// Flop the bytes on a WORD to deal with the way different
processors
	// store multi-byte numbers
	char temp;
	temp = ((char*)(&value))[0];
	((char*)(&value))[0] = ((char*)(&value))[1];
	((char*)(&value))[1] = temp;
}

void CDPXViewDoc::SwapBytes(float &value)
{
	// Flop the bytes on a "float" to deal with the way different
processors
	// store multi-byte numbers
	char temp;
	temp = ((char*)(&value))[0];
	((char*)(&value))[0] = ((char*)(&value))[3];
	((char*)(&value))[3] = temp;

	temp = ((char*)(&value))[1];
	((char*)(&value))[1] = ((char*)(&value))[2];
	((char*)(&value))[2] = temp;
}


Once determined that a swap is necessary something like this could be fired:


void CDPXViewDoc::SwapAllBytes()
{

			// The m_MagicNumber field is not swapped because it
is the flag
			// indicating that this file came from a big-endian
system
			SwapBytes(m_OffsetToImageData);
			SwapBytes(m_TotalFileSize);
			SwapBytes(m_DittoKey);
			SwapBytes(m_GenericHeaderLength);
			SwapBytes(m_IndustryHeaderLength);
			SwapBytes(m_UserHeaderLength);
			SwapBytes(m_EncryptionKey);

			SwapBytes(m_ImageOrientation);
			SwapBytes(m_ImageElements);
			SwapBytes(m_PixelsPerLine);
			SwapBytes(m_LinesPerImage);

			for(int i=0;i<8;i++)
			{
				SwapBytes(m_DataSign[i]);
				SwapBytes(m_RefLowDataCode[i]);
				SwapBytes(m_RefLowQuantity[i]);
				SwapBytes(m_RefHighDataCode[i]);
				SwapBytes(m_RefHighQuantity[i]);
				SwapBytes(m_Packing[i]);
				SwapBytes(m_Encoding[i]);
				SwapBytes(m_Offset[i]);
				SwapBytes(m_PaddingEOL[i]);
				SwapBytes(m_PaddingEOI[i]);
			}

			SwapBytes(m_XOffset);
			SwapBytes(m_YOffset);
			SwapBytes(m_XCenter);
			SwapBytes(m_YCenter);
			SwapBytes(m_XOriginalSize);
			SwapBytes(m_YOriginalSize);
			SwapBytes(m_BorderValidityXL);
			SwapBytes(m_BorderValidityXR);
			SwapBytes(m_BorderValidityYT);
			SwapBytes(m_BorderValidityYB);
			SwapBytes(m_PixelAspectH);
			SwapBytes(m_PixelAspectV);

			SwapBytes(m_FramePosition);
			SwapBytes(m_SequenceLength);
			SwapBytes(m_HeldCount);
			SwapBytes(m_FrameRate);
			SwapBytes(m_ShutterAngle);

			SwapBytes(m_SMPTETC);
			SwapBytes(m_SMPTEUB);
			SwapBytes(m_HSampling);
			SwapBytes(m_VSampling);
			SwapBytes(m_TSampling);
			SwapBytes(m_SyncToPixel);
			SwapBytes(m_Gamma);
			SwapBytes(m_BlackLevel);
			SwapBytes(m_BlackGain);
			SwapBytes(m_Breakpoint);
			SwapBytes(m_WhiteLevel);
			SwapBytes(m_Integration);
}


On an earlier question of making a DPX application fully SMPTE spec
compatible.  I'd say, forget it.  I don't think this can be done.  You'll
get to a point where too many unknowns will prevent you from making
decisions as to how to implement.  Or, at the very least, foce you to decide
from your vantage point.  You've already discovered that the header gets not
filled out, trampled, etc.  I've seen cases where you simply can't rely on
the header to show a usable picture, for example.

BTW, your FTP site doesn't seem to contain the "dpx_samples.zip" file.  I
haven't had a chance to look at any of your software yet, but I will when I
get some time.


Have fun!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Martin Euredjian
eCinema Systems, Inc.
voice: 661-305-9320
fax: 661-775-4876
martin at ecinemasys.com
www.ecinemasys.com
 
 






More information about the Tig mailing list