Developing an ImageLink Provider Script

This is intended to describe how to write a provider script.

Overview

The Plug-in enables connection to any DAM or online image resource via a provider script that manages interaction with its REST API. This is a single script containing a number of key callback functions, prefixed with cb_.

Scripts are stored in locally in the preference folder. This can be located using ImageLink section of the InDesign preferences. Scripts are implemented Lua. This is a powerful, efficient, lightweight scripting language. It supports procedural programming, object-oriented programming, functional programming, data-driven programming, and data description.

Script Operation

Configuration Options

On startup the cb_getconfiguration function for each Provider gets executed. This returns a table of properties which configure the Provider, both in terms of user interface and the underlying link subsystem within InDesign.

The key property is the scheme. This is the first component of a URI, which is used to identify which provider manages it. Its name should be unique, and confirm to URI standards. Typically it would be a DAM system name or similar. The scheme name should not start with a digit and should not contain spaces. Only one installed provider per scheme is allowed.

Storing Data

A Provider can store data for later retrieval into the preferences folder using a call to PROVIDER.setdata. This can be passed any Lua type and serialises the data for later retrieval by PROVIDER.getdata. A Typical use would be to store server location, username and password details.

Session Data

A ‘session’ is defined as the period InDesign is running, so empty on startup. A call to PROVIDER.setsession stores data for later retrieval by PROVIDER.getsession. A Typical use would be to store authentication tokens, etc.

Global Variables

Each callback is run in it’s own Lua state, so global variables cannot be used between callbacks.

Multi-Threading

Certain callbacks are made on worker threads so must be thread-safe. The majority of calls to check an assets status are made this way. All utlility functions provided by the Plug-in are thread-safe.

Status Checking

For each linked image, InDesign periodically checks its state to update its status in the Links panel. To enable this process to be efficient, URIs are batched into a single request to the cb_getassetstatus callback. Maximum batch size is limited by the batchlimit configuration settings. Calls to getassetstatus are made aysnchronously, so although the time taken to intiate a new Lua state is neglibale, a performance improvement is to be had by batching requests, even if this ends up as a single request made to the DAM.

Batching can also be controlled by the batchlimittimems settings. This specifies the maximum time that can elapse before a batch request is made. For example, if the batchlimit was 10, but only 2 requests are pending within this time, the request for 2 is made.

To allow further control, minimumcheckintervalinms can be used. A value of zero means status checking will be as often as InDesign requests, which could potentially cause loading issues on a server. Once an asset has been checked, if a subsequent check on the same URI is requested in less time than specified by this parameter, the previous asset state is reused.

Drag and Drop from an External Application

When an image is dragged from an external source, such as a Browser and dropped on an InDesign layout, the Plug-in extracts the image source (if available) and calls cb_convertexternallink. This inspects the URI and decides if it can handled it. If it can, it returns how confident it is in handling it along with a modified URI that can be placed in InDesign. All providers are checkeed and the provider with the highest confidence is chosen. If more than one provider is found, the user will be presented with a choice of which to use.

Image Downloading

The download of the images is performed by InDesign using a GET operation. In preparation cb_preparedownload is called, which fixes the URI and provides HTTP header information. Preview images use the same callback, but are downloaded by the Plug-in.

JSON Parsing

CSJON is built into the Plug-in. It provides fast, standards compliant encoding/parsing routines and full support for JSON with UTF-8, including decoding surrogate pairs.

External Modules

The preferences folder contains a require folder where external modules can be placed for use by the provider.

Scripting

External scripts can be used to connect and place images. The URI scheme should match that of a provider. E.g:

var str = '{"user":"xxx","password":"yyy"}'; 
app.httpLinkConnectionManager.httpConnect("elvis://hub.sdam.io", str);
alert(app.httpLinkConnectionManager.isConnected("elvis://hub.sdam.io"));
app.linkingPreferences.httpLinksRenditionType = LinkResourceRenditionType.FPO;
var rectframe = app.activeDocument.rectangles.add();
rectframe.geometricBounds = [10,10,30,30];
var file1 = "elvis://hub.sdam.io/file/1FfYBsucaXuBQqdXr9Ypgb/*/oNSvTtnNXEc.jpg?_=2" ;
rectframe.place(file1);
if(app.activeDocument.links[0].renditionData == LinkResourceRenditionType.FPO)
     app.activeDocument.links[0].replaceWithOriginal();

cb_scripthttpconnect is called in response to app.httpLinkConnectionManager.httpConnect.

Debugging

To aid debugging there is debug option in the preferences. This enables reloading of the script everytime the Provider menu option is selected. It also logs most operations into the logs folder within the preferences. Ensure this option is turned off during normal operation as it has an impact on performance.

Remote debugging is supported using the ZeroBrane Studio IDE.

To Debug:

• Copy mobdebug.lua and socket.lua into any location checked by the ‘require’ command (A list is shown if the module cannot be found). These are available from the ZeroBrane download.
• Launch the IDE. Go to Project | Start Debugger Server and start the debugger server (if this menu item is disabled, the server is already started)
• Add require(‘mobdebug’).start() call to your script.
• Test your script. You should see a green arrow pointing to the next statement after the start() call in the IDE and should be able to step through the code

Sample Script for Pexels

A simple script with no folder browsing or authentication. The authorisation code has been redacted:

------------------------------------------------------------------------------------------------------------------------ 
-- cb_getassets
------------------------------------------------------------------------------------------------------------------------
function cb_getassets(query_string, path, page)
   assetData =  { }

   assetData = {} 
   headers = {};
   headers["Authorization"] = " Bearer " .. "xxxxxxxxx"

   url = "https://api.pexels.com/v1/search?query=" .. HTTP.urlencode(query_string) .. "&per_page=25&page=" .. page

   code, cookies, body =  HTTP.get(url, "", headers);
   if code ~= 200 and err ~= 401 then
       error ("Invalid response from pexels [" .. code .. ", " .. body .. "]");
   end
 
   json_return = cjson.decode(body)
   if json_return == nil then error("invalid JSON in results"); end;
 
   results = json_return["photos"];
   total_hits = json_return["total_results"];
   
   total_pages = tonumber(total_hits) / 15;
   
   for x = 1, #results do
     obj = results[x];
     asset = ASSET.new(obj["src"]["original"], obj["src"]["original"], obj["src"]["small"], obj["src"]["tiny"], obj)
     table.insert(assetData, asset );
   end
   return total_pages, assetData; 
end

------------------------------------------------------------------------------------------------------------------------
-- cb_getassetstatus
------------------------------------------------------------------------------------------------------------------------
function cb_getassetstatus(uri_list)
  return_list = {} 
  for id, uri in pairs(uri_list) do
    assetData =  { }
    uri:setcomponent('scheme', 'http');
    size, datetime = HTTP.getinfo(uri:get());
    if size == nil then
        assetData["status"] =  "unavailable";
    else
        assetData["size"] =  size
        assetData["status"] =  "available";
    end
    table.insert(return_list, assetData );
  end
     
  return return_list;
end

------------------------------------------------------------------------------------------------------------------------
-- cb_preparedownload
------------------------------------------------------------------------------------------------------------------------
function  cb_preparedownload(uri)
  headers = {}
  headers["Authorization"] = " Bearer " .. "xxxxxxxxx"
  uri:setcomponent("scheme", "https");
  return uri, headers;
end

------------------------------------------------------------------------------------------------------------------------
-- cb_renderassetinfo
------------------------------------------------------------------------------------------------------------------------
function cb_renderassetinfo(asset, renderer)
  local metadata = asset:getmetadata();
  renderer:setrgbcolor(0.2,0.2,0.2);
  renderer:fillrect(0,0,renderer:width(),renderer:height());
  local original = metadata.src.original;
  renderer:drawstring(metadata.photographer, 4, 12, renderer:width()-4, "left", "white");
  renderer:drawstring(metadata.width .. " x " .. metadata.height, 4, 12+15, renderer:width()-4, "left", "white");
end

------------------------------------------------------------------------------------------------------------------------
-- cb_getconfiguration
------------------------------------------------------------------------------------------------------------------------
function cb_getconfiguration()
    c = {
        login       = false,
        fpo         = false,
        original    = false,
        embed       = true,
        headersize  = 35,
        description = "Stunning free images & royalty free stock.",
        url         = "https://www.pexels.com",
        scheme      = "pexels",
        name        = "Pexels",
        version     = "1.0.1",
        infotext    = "Click to learn more",
        infolink    = "https://www.pexels.com",
        batchlimit  = 10,
        timeinms    = 1000
   };
   return c;
end

Sample Script For Our Demo DAM

A more complex example incorporating authentication, folder browsing and FPO image support.

------------------------------------------------------------------------------------------------------------------------
-- Callbacks are a core component of the Plug-in and act as glue between the Plug-in and the external API.
--
-- Execution of each callback is made in it's own Lua state. Certain functions such as <b>cb_getassetstatus</b> are called aysnchronously.
-- <p>Sequence of callbacks after login is selected:
-- <li><b>cb_login</b></li>
-- <li><b>cb_connect</b> (if PROVIDER.canconnect)</li>
-- <li><b>cb_getfolders</b> (if exists and PROVIDER.isconnected)</li>
-- <li><b>cb_getassets</b> (if PROVIDER.isconnected)</li>
--
--
-- @module Callbacks
------------------------------------------------------------------------------------------------------------------------


--- Status of an Asset.
-- @table AssetStatus
-- @int size Size in Bytes
-- @string status "available", "inaccessible" or "missing"
-- @string date Modification timestamp (DATETIME)
-- @string version Version information (optional)


--- Describes a Folder. A folder is any path that can contain Assets.
-- @table Folder
-- @string name folder name
-- @string path folder path

--- Properties of the provider.
-- @table Properties
-- @string scheme URI scheme. This should be unique amongst all providers and conform the URI standards.
-- @string url URL displayed in the panel.
-- @string version Version string in the format X.X.X
-- @string infotext Text displayed in the panel
-- @string infolink Link when infotext is clicked on
-- @int batchlimit Batch Limit for Asset status checks
-- @int batchlimittimems Maximum time for a batch operation in Ms
-- @int minimumcheckintervalinms Minimum Asset status check period in Ms
-- @bool logon Requires login
-- @bool fpo Supports For Position Only images
-- @bool original Supports Original images
-- @number[opt] headersize Asset information area depth

------------
-- Called when the Login menu is selected. The login menu is only enabled when PROVIDER.canconnect() is false.
-- Determines if it's ok to attempt a connection and assigns a result to PROVIDER.setcanconnect().
-- Typically presents a login dialog box and stores information for use when connecting.

function cb_login()
    local username = ""
    local password = ""
    local server = ""

    PROVIDER.setcanconnect(false);
    obj = PROVIDER.getdata();
    if obj ~= nil then
      if obj.user ~= nil then username = obj.user end;
      if obj.password ~= nil  then password = obj.password end;
      if obj.server ~= nil  then server = obj.server end;
    end

    username, password, server = showLoginDialog(username, password, server);

    if username ~= nil and password ~= nil and server ~= nil then

      stored_data = {};
      stored_data.user = username;
      stored_data.password = password;
      stored_data.server = server;

     PROVIDER.setdata(stored_data);

     PROVIDER.setcanconnect(true);
    end
end

------------------------------------------------------------------------------------------------------------------------
-- Called prior to any request to create a physical connection if not connected. Should called PROVIDER.setisconnected()
-- and persist data required for subsequent requests. Typically used to persist session data.

function cb_connect()
    local setting = PROVIDER.getdata();
    if setting == nil then return end;
    if setting.user == nil then return end

    login_url = setting.server .. "/services/login?returnProfile=true&" .. "username="  .. setting.user .. "&password=" .. setting.password

    code, cookies, body =  HTTP.post(login_url, "")

    if code ~= 200 and err ~= 401 then
        error("Server returned error " .. code);
    end
    json_return = cjson.decode(body)
    if json_return == nil then
         error("Invalid response from server:" .. body);
    end

    if json_return["loginSuccess"] ~= true then
         error("Login failed :" .. json_return["loginFaultMessage"]);
       PROVIDER.setcanconnect(false);
    end
    -- store the login token in the session data
    conn_settings = {}
    conn_settings["cookies"] = cookies;
    conn_settings["csrfToken"] = json_return["csrfToken"];

   PROVIDER.setsessiondata(conn_settings);
   PROVIDER.setisconnected(true);
end

------------
-- Get a table of Folders at the given path. If this function does not exist, the widget is not shown.
-- @tparam string path Folder path
-- @treturn table Table of Folders

function cb_getfolders(path)

  assetData =  { }

  local urlpath = HTTP.urlencode(path);

  local connectionSettings = PROVIDER.getsessiondata();

  local setting = PROVIDER.getdata();

   -- build request URL - needs to be built from connection settings....
   url = setting.server .. "/services/browse?path=" .. urlpath .. "&includeFolders=true&includeAssets=false"

   headers = { };

   local cookies = connectionSettings["cookies"];

   headers["cookie"] = "authToken=" .. cookies["authToken"];
   headers["X-CSRF-TOKEN"] = connectionSettings["csrfToken"];

   code, cookies, body =  HTTP.post(url, "", headers);

    if code ~= 200 and err ~= 401 then
        error("server returned error code" .. code .. " " .. body);
        return assetData;
    end

    json_return = cjson.decode(body)

  for i = 1, #json_return do
    item = json_return[i];
    if item ~= nil then
      if item["directory"] == true then
        obj = {};
        obj["path"] = item["assetPath"];
        obj["name"] = item["name"];
        table.insert(assetData, obj);
      end
    end
  end
  return assetData;
end

------------------------------------------------------------------------------------------------------------------------
-- Get a table of Assets. This is the main function used to populate the panel and determine paging.
-- @tparam string query_string search string enterered into panel
-- @tparam string path path to location to assets to query.
-- @tparam integer page page number to retrieve.
-- @treturn integer Total number of pages available.
-- @treturn Table table of new Assets.


function cb_getassets(query_string, path, page)
   assetData =  { }

  -- build request URL - needs to be built from connection settings....
  local setting = PROVIDER.getdata();
 
  url = setting.server .. "/services/search";

   local params = "num=1000&q=";

   if query_string:len() > 0 then
       local queryBit  = "(" .. query_string .. ")";
       params = params .. HTTP.urlencode(queryBit);
   end


   if path:len() > 0 then

      if query_string:len() > 0 then
        local andBit = " AND ";
        params = params .. HTTP.urlencode(andBit);
      end

      local ancestorPathsBit = "ancestorPaths:\"" .. path .. "\"";
      params = params .. HTTP.urlencode(ancestorPathsBit);

    end

   headers = { };
   cookies = PROVIDER.getsessiondata()["cookies"];
   headers["cookie"] = "authToken=" .. cookies["authToken"];

   code, cookies, body =  HTTP.post(url, params, headers);

   PROVIDER.logfile("body.txt", body);

    if code ~= 200 and err ~= 401 then
       PROVIDER.setisconnected(false);
        error ("failed with error " .. code )
    end

    json_return = cjson.decode(body)
    if json_return == nil then
       PROVIDER.setisconnected(false);
        error ("failed to decode json")
   end;

    hits = json_return["hits"];
    if hits == nil then return assetData; end;


    for k,this_hit in pairs(hits) do
        asset = ASSET.new(this_hit["metadata"].filename,this_hit["originalUrl"],this_hit["previewUrl"],this_hit["thumbnailUrl"], this_hit["metadata"])
       table.insert(assetData, asset );
    end
    total_pages = 1;

  return total_pages, assetData;
end


function showLoginDialog(pUsername, pPassword, pServer)

  -- create dialog
  dialogInfo = { title = "Logon To DAM", width = "350", height = "140"};
  myDialog = DIALOG.new(dialogInfo);

  -- add controls
  ctrl1 = { type = "statictext", id = "static3", title = "Server:", left = 20, top = 12, width = 80, heigth = 20 };
  myDialog:addwidget(ctrl1);

  ctrl2 = { type = "statictext", id = "static1", title = "User Name:", left = 20, top = 40, width = 80, heigth = 20 };
  myDialog:addwidget(ctrl2);

  ctrl3 = { type = "statictext", id = "static2", title = "Password:", left = 20, top = 68, width = 80, heigth = 20 };
  myDialog:addwidget(ctrl3);


  
  ctrl4 = { type = "editbox", id = "url", left = 105, top = 12, width = 220, heigth = 20, content = pServer };
  myDialog:addwidget(ctrl4);

  ctrl5 = { type = "editbox", id = "name", left = 105, top = 40, width = 220, heigth = 20, content = pUsername };
  myDialog:addwidget(ctrl5);

  ctrl6 = { type = "editboxpassword", id = "password", left = 105, top = 68, width = 220, heigth = 20, content = pPassword };
  myDialog:addwidget(ctrl6);



  ctrl7 = { type = "cancelbutton", title = "Cancel", id = "cancel1", left = 150, top = 105, width = 80, heigth = 20,  };
  myDialog:addwidget(ctrl7);

  ctrl8 = { type = "okbutton", title = "Ok", id = "ok", left = 240, top = 105, width = 80, heigth = 20 };
  myDialog:addwidget(ctrl8);

  -- open the dialog
  if myDialog:open() then
    nameWidget = myDialog:getwidget("name");
    passwordWidget = myDialog:getwidget("password");
    urlWidget = myDialog:getwidget("url");
    return nameWidget.content, passwordWidget.content, urlWidget.content;
  end

  return nil, nil, nil;
end

------------------------------------------------------------------------------------------------------------------------
-- Get AssetStatus for the given list of URI's.
-- This function is called for batches of URI's based the batchlimit. Called asynchronusly to update the links panel.
-- @tparam table uri_list list of URI
-- @treturn table Table of AssetStatus, matching the size of the parameter list

function cb_getassetstatus(uri_list)

  local connectionSettings = PROVIDER.getsessiondata();
  if connectionSettings == nil then return end;
  local cookies = connectionSettings["cookies"];

  -- Set the cookie in a header
  local headers = { };
  headers["cookie"] = "authToken=" .. cookies["authToken"];


  return_list = {}
  for id, uri in pairs(uri_list) do
       table.insert(return_list, GetSingleAssetMetaData(uri, headers) );
  end

  return return_list;
end


function GetSingleAssetMetaData(uri, post_headers)
  assetData =  { }

  uri_str = uri:get();
  -- extract the ID from the URL
  asset_id = uri_str:match(".*file/(.*)/%%2A");
  if asset_id == nil then
    asset_id = uri_str:match(".*thumbnail/(.*)/%%2A");
  end;
  if asset_id == nil then
    asset_id = uri_str:match(".*preview/(.*)/%%2A");
  end;

  if asset_id == nil then 
      return assetData; 
  end;

  -- build request URL

   local setting = PROVIDER.getdata();

   url = setting.server .. "/services/search?q=" .. asset_id

   code, cookies, body =  HTTP.post(url, "", post_headers);
     if code ~= 200 and err ~= 401 then
        PROVIDER.logerror("server returned error " .. " with response " .. body);
        PROVIDER.setisconnected(false);
        return assetData;
    end

    json_return = cjson.decode(body)
    if json_return == nil then 
        PROVIDER.logerror("invalid JSON in body " .. body);
        PROVIDER.setisconnected(false);
        return assetData; 
    end;

    hits = json_return["hits"];
    if hits == nil then
       PROVIDER.logerror("Response is malformed");
       PROVIDER.setisconnected(false);
       return assetData;
    end


    first_hit = hits[1];
    if first_hit == nil then 
      assetData["status"] =  "missing";
      return assetData; 
    end;


    metadata = first_hit["metadata"];

    if metadata == nil then 
        return assetData; 
    end;

    fileSize = metadata["fileSize"];
    if fileSize == nil then 
        return assetData; 
    end;

    -- modified date
    --modified = metadata["xmpModifyDate"];
    --modified_formatted = modified["formatted"];

    -- parse a time like this 2019-08-05 12:50:41+0300
    --local inYear, inMonth, inDay, inHour, inMinute, inSecond = string.match(modified_formatted, '^(%d%d%d%d)-(%d%d)-(%d%d) (%d%d):(%d%d):(%d%d) ')

    inYear = 2020
    inMonth = 1
    inDay = 1
    inHour = 1
    inMinute = 1
    inSecond= 1

    -- Return the meta data
    assetData["size"] =  tonumber(fileSize["value"]);
    assetData["status"] =  "available";
    --assetData["modified"] = TIME.new(inYear, inMonth, inDay, inHour, inMinute, inSecond);
    --assetData["version"] = metadata["versionNumber"];
    return assetData;

end

------------------------------------------------------------------------------------------------------------------------
-- Called before an image is download to provide POST header data and URI correction.
-- @tparam URI uri The source URI
-- @treturn URI Resolved URI. Typically would involve correcting the scheme
-- @treturn table Table of POST header information

function  cb_preparedownload(uri)
    -- Change the scheme to https
    uri:setcomponent("scheme", "https");
    cookies = PROVIDER.getsessiondata()["cookies"];

   -- construct post headers
   post_headers = { };
   post_headers["cookie"] = "authToken=" .. cookies["authToken"];
   return uri, post_headers
end




------------------------------------------------------------------------------------------------------------------------
-- Returns the rendition type of passed URI.
-- @tparam URI uri
-- @treturn string the rendition type, either "fpo" or "original"

function cb_getrenditiontype(uri)
  uri_str = uri:get();
  -- if the uri contains preview, it's the pfo
  asset_id = uri_str:match(".*preview/(.*)/");
  if asset_id  ~= nil then
    return "fpo"
  end
  return "original"
end

------------------------------------------------------------------------------------------------------------------------
-- Fetches the URI of the input asset with given rendition.
-- @tparam URI uri
-- @tparam string type the rendition type for which URI is required, either "fpo" or "original"
-- @return URI of given the rendition type


function cb_getasseturi(uri, type)
--  require('mobdebug').start()
  uri_str = uri:get();

  if rendertype == "original" then
    uri_str = string.gsub(uri_str, "preview/", "file/");

  else
    uri_str = string.gsub(uri_str, "file/", "preview/");
  end

  return URI.new(uri_str);
end




------------
-- Draw the information under the preview.
-- @tparam Asset asset Asset to draw
-- @tparam RENDERER renderer Provides drawing functions

function cb_renderassetinfo(asset, renderer)
    local metadata = asset:getmetadata();

    renderer:setrgbcolor(0.4,0.4,0.4);
    renderer:fillrect(0,0,renderer:width(),renderer:height());
    renderer:setrgbcolor(1,0,0);

    renderer:drawstring(asset:getname(), 2, 10, renderer:width()-4, "left", "white");

    renderer:setrgbcolor(0,1,0);
    renderer:drawtick(10,20,5);
    renderer:drawstring(metadata.downloadCount, 10, 24, renderer:width()-12, "right");
end



------------
-- Get the configurations settings of the provider.
-- @treturn Properties Table of settings.

function cb_getconfiguration()
    c = {
        url        = "https://www.65bit.com",
        scheme      = "bitdam",
        name       = "Generic DAM",
        version    = "1.0.0",
        infotext    = "Click to learn moreX",
        infolink    = "https://65bit.com",
        batchlimit  = 10,
        timeinms    = 1000
   };
   return c;
end




function cb_convertexternallink(externaluri) 
    -- will throw an error if any extracted part fails, 
    -- which is okay as this provider will then not be used
    -- to place the image.
    asset_id = externaluri:match(".*preview/(.*)/previews/");
    filename,ext  = externaluri:match(".*/(.*)_preview(.*?_=)");
    uri_str = "https://hub.sdam.io/file/" .. asset_id .. "/%2A/" .. filename .. ext .. "2";
    -- return 999 = we're certain this URI belongs to us
    -- return uri_str = the actual URI to place
    return 999, uri_str;
end


function cb_scripthttpconnect(params) 
     PROVIDER.setdata(cjson.decode(params));
     PROVIDER.setcanconnect(true);
end

The post Developing an ImageLink Provider Script appeared first on 65bit Software.