Send binary data to webhook

[badasukerubin]

The problem I am facing
I'm using the Webhook extension which correctly sends the document payload (content in json) to my given webhook URL.

The solution I would like
I would like also to send binary data along with the json data to store directly in the db

Alternatives I have considered

1. Storing json in DB. This doesn't work correctly as whenever I restart hocus-pocus, it appends the incoming data (in onLoadDocument)

Additional context
I convert the document from json to plain text on the server, which is saved to the file system for my users that want to download plain text, I'd like to also save the binary data to enable history merge/update for edit inside my app.

const server = Server.configure({
  name: config.name,
  port: Number(config.port),
  extensions: [
    /** @ts-ignore */
    new Logger(),
    /** @ts-ignore */
    new Webhook({
      events: [
        Events.onConnect,
        Events.onCreate,
        Events.onChange,
        Events.onDisconnect,
      ],
      url: 'http://app.test/api/test-webhook',
      transformer: TiptapTransformer.extensions([Document, Paragraph, Text]),
    }),
  ],
  onAuthenticate,
  onLoadDocument,
});

export const onLoadDocument = async (data: onLoadDocumentPayload) => {
  const { documentName, document, requestParameters } = data;

  const resourceRoute = requestParameters.get('resourceRoute');

  if (!resourceRoute) {
    return document;
  }

  try {
    const response = await fetch(resourceRoute);

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const textData = await response.text();

    if (!textData) {
      throw new Error('No text data found');
    }

    const jsonData = generateJSON(textData, [Document, Paragraph, Text]);

    const ydoc = TiptapTransformer.toYdoc(jsonData, 'default', [
      Document,
      Paragraph,
      Text,
    ]);

    Y.applyUpdate(document, Y.encodeStateAsUpdate(ydoc));

    // return ydoc;
  } catch (error) {
    console.error('Error processing document:', error);
    return document;
  }
};

[michaelsieminski]

Hi, you definitely need the Database Extension to store the Binary data on your database! Check this out https://tiptap.dev/docs/hocuspocus/server/extensions#database

[janthurau]

This looks like a pretty custom setup ; you need to make sure that you save and load the same yjs data, otherwise you get duplicated content (as you noticed). Feel free to extend the webhook extension to push / pull the data when needed (onStoreDocument / onLoadDocument). As long as you store + pull yjs binary, it will work.