Administration Guide : How Xinet Reads Data from Files : XMP Metadata and Xinet

XMP Metadata and Xinet
The Extensible Metadata Platform (XMP), a rich-media format developed by Adobe, provides another way to embed data about a file into the file itself. In that regard, it is similar to IPTC metadata that Xinet has supported since version 3.01. However, XMP is a more flexible protocol with all the older IPTC fields subsumed within it. It contains:
Entering XMP metadata
Some files enter workflows with metadata already embedded. For example, raw files from digital cameras may provide information about file size and format, camera settings, lighting conditions, etc. When an operator saves a file containing this data, or, adds data by hand using an application that creates an XMP packet, the application automatically converts this information into XMP data. This conversion is essential for Xinet, because otherwise, the data may not be in a format that the Xinet database understands.
Xinet can read XMP data embedded in files and in turn, write XMP information into those files which support embed XMP packets. For files that cannot embed XMP packets—for example, Microsoft Office files—it is still possible to maintain XMP information within Xinet. You can associate XMP data with a file inside the database, and view that information inside Pilot. When Pilot downloading such a file, users may request that any associated XMP data also be downloaded in a “sidecar” file. Downloading associated XMP information from Xinet provides more information.
XMP and Xinet
The Xinet server automatically populates its database with all standardized XMP data in files saved on Xinet enabled volumes. It is also possible to make custom XMP panels on the client side and corresponding data fields on the Xinet server. Xinet reads XMP data from files whenever:
The dblogd(8) daemon either updates information about a file or adds the file to the database, for example, when a file arrives on the server or is modified.
The syncvoltodb(8) program runs.
The syncxmp program runs
With more information in the database, the Xinet GUI has been modified so that administrators can view Data Field information “by sets.”
Predefined Data Field sets include:
Displays all datafields, including locked fields and custom, user-defined fields
Displays only IPTC and XMP fields created during installation.
Displays only custom, nativeadmin-defined fields, which might include custom XMP fields
The Xinet administrator can also create other Data Field sets. (See Establishing Data Field Sets for details about creating a custom set.) Also note that Figure  also shows an XMP-generated data field type, IntMap (a shortened version of Integer Map). IntMap values are actually integers that have text labels associated with some or all of the possible values, e.g., Urgency.
Once XMP data has been saved in the database, it can be searched, viewed and even edited through a standard browser.
Caution: Custom XMP fields in a workflow
Data entered by operators using the File Info or Document Metadata panels in Adobe CS applications will be embedded within files. This information will become visible to Xinet users with proper access permissions as soon as a file in which metadata has been embedded has been saved on the server.
In some scenarios, XMP fields might be created with Xinet which do not have corresponding client-side custom panel fields. In such instances, Adobe CS operators will not be able to fill in values; however, Xinet users with proper access permissions could see and also possibly edit values. Metadata that entered the system through such a field would be embedded in the file itself. In some workflows, it might be advantageous to set them up this way; disastrous in others. Generally, sites want the same XMP fields available in Xinet for all Adobe products.
Quick start guide for adding custom fields
Xinet will automatically read standard XMP metadata from files. It can also read custom metadata written in the XMP format. To make use of client-side custom fields in Xinet, you must also create them on the server side. Here is an overview of what
is involved:
On the client side, using a text editor or third-party application, create a custom XMP panel. (Refer to Xinet TechNote 0295 available at if you need creating custom XMP FileInfo panels for Adobe CS 3 (or later) applications.)
Background: Adding custom XMP Xinet fields
For custom fields in client-side XMP panels, the nativeadmin administrator must also create corresponding fields in the Xinet database that will hold metadata introduced from a custom panel. For general information about creating data fields in Xinet, refer to Working with Xinet Data Fields. When creating fields that correspond to the XMP namespace, keep in mind:
Each keyword must match its appearance in the corresponding XMP code. Case-sensitivity withstanding, they must match exactly.
Xinet data field names can not contain blank spaces. This is a limitation from XMP specifications, not from Xinet. If the Xinet fields contain blank spaces, they will not be populated with XMP data.
The data field type, e.g., pop-up, boolean, etc., must be the same in Xinet as is in the custom panel.
XMP conventions maintain that, internally, data field names must follow the conventions mentioned in items 1. and 2., and in addition, be prepended by a namespace. Internally, Xinet checks whether or not you have prepended fields you define with a namespace and if you haven’t, prepends xwnv. This means that you could, for example, name a field Approval, yet internally, the database would use xwnv:Approval. Xinet also offers some other alternatives if you want to display labels for users that violate XMP regulations. For example, perhaps you want them to see the phrase Approval Status, which contains a forbidden blank space.
The easiest work-around is to use the local.js variable field to define a label.
Downloading associated XMP information from Xinet
Any XMP information that can be viewed on the Web can also be downloaded. By default, when a remote Xinet user downloads a file, that file will contain any XMP information embedded in the file, but will not contain XMP data that is merely associated with the file in the database.
Some workflows may be interested in exporting all XMP data associated with a file. To that end, the Xinet Collection includes an option that allows a user to export all XMP data.
When a user marks the XMP Export option Yes, the file itself and an additional file containing all the XMP data will download. For example, when downloading 192547-b.tiff, Xinet downloads 192537-b.tiff and 192537-b.xmp:
More generally stated, downloading the high-resolution version of a file from the Collection called myfile.extension would result in two files on the desktop:
myfile.extension and myfile.xmp
Downloading the low-resolution version of myfile.extension would result in only a low-resolution copy of myfile.extension. XMP data will not be included.
Understanding data field names
Labels for fields in the Xinet database may not always have the exact labels operators see in Adobe CS dialogs. This has to do with Adobe’s rules for aliases between namespaces.
Looking inside an Adobe CS File Info panel, you’ll notice a list of metadata “namespaces.” They include (as of this publication date) Description, Camera Data 1, Camera Data 2, Categories, History, IPTC Contact, IPTC Content, IPTC Image, IPTC Status, Adobe Stock Photos, Origin, Dublin Core Properties, XMP Core Properties, XMPMedia Management Properties, PDF Properties, PNG Properties, and TIFF Properties.
When an operator saves a file, the Adobe application actually rewrites a number of standard metadata formats into XMP. Often times, then, the same data will be associated with a file more than once. For example, an operator may enter data in IPTC-specified fields and it will appear again, encoded in XMP format. XMP specifications provide aliases for many such fields, sometimes affixing a different XMP label for a given metadata field than the label affixed to that field in its original namespace.
For maximum backward compatibility, Xinet is preserving the old name used for IPTC-NAA fields in Photoshop 7 and lower. Below shows possible mappings of Photoshop 8 and higher to the old-style IPTC fields used in Xinet.
For more information, about the aliases specified in XMP rules and adopted by Xinet, please refer to Adobe’s documentation, XMP – Extensible Metadata Platform, which you can download from:
Another good source of information:
Background: How Metadata is stored in the Xinet database
Let’s start with an example file, gears.tiff, that has the following fields, assigned with the following values:
The gears.tiff file’s fileid is 374. (You can determine this by looking in its source when browsing or when displaying information about the file using Xinet’s imageinfo program.)
Each field created has a record in the keyword table. They are distinguished by keywordids. You can find the keywordid for each record in at least two ways:
If you were to click on the file’s Xinet Image Info icon and then look at its source code, you would find:
var keywords = new Array();
keywords[0] = new
keywords[1] = new
keywords[2] = new
keywords[3] = new keyworditem(60,3,1,"a_integer",'',3,0,14,1,0,0,'',0,0,0,0,0);
keywords[4] = new keyworditem(58,4,1,"a_text",'',253,64,14,1,0,0,'',0,0,0,0,0);
keywords[5] = new
keywords[6] = new
The first item in the parentheses is the keywordid number for the field.
The fourth item in the parentheses is the name of the keyword.
If you were to provide the MySQL client with:
select keywordid, name from keyword;
the results should include:
58 | a_text
59 | a_date
60 | an_integer
61 | a_float
62 | a_boolean
63 | a_text_popup
64 | an_integer_popup
The keyword table also specifies the type of values that it can hold, its appearance, etc. See Xinet TechNote 137: Documentation for Xinet Venture MySQL data tables for full details.
Every field in the keyword table has a corresponding field in the keyword1 table. This is where the current value of a field is stored. In particular, the keywordid number has a corresponding FieldXX field in keyword1, where XX is the keywordid number. For instance, the keyword with keywordid number 58 (a_text) has an associated field in keyword1 called Field62.
Any file that has metadata applied to it has an entry in the keyword1 table. The primary field in this table is the fileid. Note that only files with metadata applied to them are in this table. For every fileid in this table, a value can optionally be applied to any of the Field fields.
In the above example, fileid 374 has values in 7 fields within keyword1.
Here is what they look like in MySQL:
If the field is a popup, the applied value shows up in the keyword1 table, but the popup values themselves are stored in one of many value tables. Popup values for text fields are stored in value253, while integer values are stored in value2. The number refers to the data type of the values stored. (Again, see Xinet Technote 137 for full details.)
In the above example, the popup values (5/10/15) for the an_integer_popup keyword are stored in the value2 table. When a value of 10 is assigned to the gears.tiff file, the value of 10 is stored in the keyword1 table.