Skip to content

fix: normalize email addresses when retrieving from server #11055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: normalize email addresses when retrieving from server #11055

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions lib/Address.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,21 @@ private function __construct(Horde_Mail_Rfc822_Address $wrapped) {
$this->wrapped = $wrapped;
}

public static function fromHorde(Horde_Mail_Rfc822_Address $horde): self {
public static function fromHorde(Horde_Mail_Rfc822_Address $horde, bool $normalize = false): self {
if ($normalize) {
return self::fromRaw($horde->personal, $horde->bare_address, $normalize);
}
return new self($horde);
}

public static function fromRaw(string $label, string $email): self {
$wrapped = new Horde_Mail_Rfc822_Address($email);
public static function fromRaw(?string $label, string $email, bool $normalize = false): self {
if ($normalize) {
$wrapped = new Horde_Mail_Rfc822_Address(self::normalizeAddress($email));
} else {
$wrapped = new Horde_Mail_Rfc822_Address($email);
}
// If no label is set we use the email
if ($label !== $email) {
if ($label !== null && $label !== $email) {
$wrapped->personal = $label;
}
return new self($wrapped);
Expand Down Expand Up @@ -117,4 +124,13 @@ public function equals($object): bool {
return $this->getEmail() === $object->getEmail()
&& $this->getLabel() === $object->getLabel();
}

private static function normalizeAddress(string $address): string {
// remove single quotes and whitespace the might exist
// Examples:
// [email protected]
// '[email protected]'
// ' [email protected]'
return strtolower(trim(trim($address, "'")));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned at #10980 (comment).

Example:

if (str_starts_with($address, "'") && str_ends_with($address, "'")) {
	$address = substr($address, 1, -1); // or trim($address, "'"); whatever you prefer
}
return strtolower(trim($address));

Given that we need the same logic in the repair job, I'd suggest moving the normalization to a small helper or utility service to make it reusable and more testable.

}
}
6 changes: 3 additions & 3 deletions lib/AddressList.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ public static function parse($str) {
* @param Horde_Mail_Rfc822_List $hordeList
* @return AddressList
*/
public static function fromHorde(Horde_Mail_Rfc822_List $hordeList) {
$addresses = array_map(static function (Horde_Mail_Rfc822_Address $addr) {
return Address::fromHorde($addr);
public static function fromHorde(Horde_Mail_Rfc822_List $hordeList, bool $normalize = false): self {
$addresses = array_map(static function (Horde_Mail_Rfc822_Address $addr) use ($normalize) {
return Address::fromHorde($addr, $normalize);
}, array_filter(iterator_to_array($hordeList), static function (Horde_Mail_Rfc822_Object $obj) {
// TODO: how to handle non-addresses? This doesn't seem right …
return $obj instanceof Horde_Mail_Rfc822_Address;
Expand Down
58 changes: 58 additions & 0 deletions lib/BackgroundJob/RepairRecipients.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
<?php

declare(strict_types=1);

/**
* SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
* SPDX-License-Identifier: AGPL-3.0-or-later
*/

namespace OCA\Mail\BackgroundJob;

use OCP\AppFramework\Utility\ITimeFactory;
use OCP\BackgroundJob\IJobList;
use OCP\BackgroundJob\TimedJob;
use OCP\DB\QueryBuilder\IQueryBuilder;
use OCP\IDBConnection;

class RepairRecipients extends TimedJob {

public function __construct(

Check warning on line 20 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L20 

Added line #L20 was not covered by tests
protected ITimeFactory $time,
private IDBConnection $db,
private IJobList $jobService,
) {
parent::__construct($time);
$this->setInterval(300);

Check warning on line 26 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L25-L26 

Added lines #L25 - L26 were not covered by tests
}

protected function run($argument): void {

Check warning on line 29 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L29 

Added line #L29 was not covered by tests
// fetch all quoted emails
$select = $this->db->getQueryBuilder();
$select->select('id', 'email')
->from('mail_recipients')
->where(
$select->expr()->like('email', $select->createNamedParameter('\'%\'', IQueryBuilder::PARAM_STR))
)
->setMaxResults(1000);
$recipients = $select->executeQuery()->fetchAll();

Check warning on line 38 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L31-L38 

Added lines #L31 - L38 were not covered by tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$recipients = $select->executeQuery()->fetchAll();
$result = $select->executeQuery();
$recipients = $result->fetchAll();
$result->closeCursor();

// update emails
$update = $this->db->getQueryBuilder();
$update->update('mail_recipients')
->set('email', $update->createParameter('email'))
->where($update->expr()->in('id', $update->createParameter('id'), IQueryBuilder::PARAM_STR));
foreach ($recipients as $recipient) {
$id = $recipient['id'];
$email = $recipient['email'];
$email = trim(str_replace('\'', '', (string)$email));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$email = trim(str_replace('\'', '', (string)$email));
$email = trim(trim($email, "'"));

The best approach would be to reuse the logic from the address class. If we don't go with the helper/utility, we should still use trim to remove the first and last character (even if we can be very sure, due to the query, that we're only dealing with emails starting and ending with a ').

$update->setParameter('id', $id, IQueryBuilder::PARAM_STR);
$update->setParameter('email', $email, IQueryBuilder::PARAM_STR);
$update->executeStatement();

Check warning on line 50 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L40-L50 

Added lines #L40 - L50 were not covered by tests
}
// remove job depending on the result
if ($recipients === []) {
$this->jobService->remove(RepairRecipients::class);

Check warning on line 54 in lib/BackgroundJob/RepairRecipients.php

View check run for this annotation

Codecov / codecov/patch

lib/BackgroundJob/RepairRecipients.php#L53-L54 

Added lines #L53 - L54 were not covered by tests
}
}

}
10 changes: 5 additions & 5 deletions lib/IMAP/ImapMessageFetcher.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,11 +250,11 @@ public function fetchMessage(?Horde_Imap_Client_Data_Fetch $fetch = null): IMAPM
$this->uid,
$envelope->message_id,
$fetch->getFlags(),
AddressList::fromHorde($envelope->from),
AddressList::fromHorde($envelope->to),
AddressList::fromHorde($envelope->cc),
AddressList::fromHorde($envelope->bcc),
AddressList::fromHorde($envelope->reply_to),
AddressList::fromHorde($envelope->from, true),
AddressList::fromHorde($envelope->to, true),
AddressList::fromHorde($envelope->cc, true),
AddressList::fromHorde($envelope->bcc, true),
AddressList::fromHorde($envelope->reply_to, true),
$this->decodeSubject($envelope),
$this->plainMessage,
$this->htmlMessage,
Expand Down
28 changes: 28 additions & 0 deletions lib/Migration/Version5000Date20250507000000.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<?php

declare(strict_types=1);

/**
* SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
* SPDX-License-Identifier: AGPL-3.0-or-later
*/

namespace OCA\Mail\Migration;

use Closure;
use OCP\BackgroundJob\IJobList;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;

class Version5000Date20250507000000 extends SimpleMigrationStep {

public function __construct(

Check warning on line 19 in lib/Migration/Version5000Date20250507000000.php

View check run for this annotation

Codecov / codecov/patch

lib/Migration/Version5000Date20250507000000.php#L19 

Added line #L19 was not covered by tests
private IJobList $jobService,
) {
}

Check warning on line 22 in lib/Migration/Version5000Date20250507000000.php

View check run for this annotation

Codecov / codecov/patch

lib/Migration/Version5000Date20250507000000.php#L22 

Added line #L22 was not covered by tests

public function postSchemaChange(IOutput $output, Closure $schemaClosure, array $options) {
$this->jobService->add(\OCA\Mail\BackgroundJob\RepairRecipients::class);

Check warning on line 25 in lib/Migration/Version5000Date20250507000000.php

View check run for this annotation

Codecov / codecov/patch

lib/Migration/Version5000Date20250507000000.php#L24-L25 

Added lines #L24 - L25 were not covered by tests
}

}
4 changes: 2 additions & 2 deletions lib/Service/PhishingDetection/PhishingDetectionService.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public function checkHeadersForPhishing(Horde_Mime_Headers $headers, bool $hasHt
$customEmail = null;
$fromHeader = $headers->getHeader('From');
if ($fromHeader instanceof Horde_Mime_Headers_Element_Address) {
$firstAddr = AddressList::fromHorde($fromHeader->getAddressList(true))?->first();
$firstAddr = AddressList::fromHorde($fromHeader->getAddressList(true), true)->first();
$fromFN = $firstAddr?->getLabel();
$fromEmail = $firstAddr?->getEmail();
$customEmail = $firstAddr?->getCustomEmail();
Expand All @@ -45,7 +45,7 @@ public function checkHeadersForPhishing(Horde_Mime_Headers $headers, bool $hasHt
if ($replyToHeader instanceof Horde_Mime_Headers_Element_Address) {
$replyToAddrs = $replyToHeader->getAddressList(true);
if (isset($replyToAddrs)) {
$replyToEmail = AddressList::fromHorde($replyToAddrs)->first()?->getEmail();
$replyToEmail = AddressList::fromHorde($replyToAddrs, true)->first()?->getEmail();
}
}

Expand Down
32 changes: 32 additions & 0 deletions tests/AddressTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,36 @@ public function testDoesNotEqualBecauseDifferentLabel() {

$this->assertFalse($equals);
}

public function testNormalizedWithSingleQuotes() {
$address = Address::fromRaw(null, "'[email protected]'", true)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);

$address = Address::fromHorde(new Horde_Mail_Rfc822_Address("'[email protected]'"), true)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);
}

public function testUnnormalizedWithSingleQuotes() {
$address = Address::fromRaw(null, "'[email protected]'", false)->toHorde();
$this->assertEquals("'[email protected]'", $address->bare_address);

$address = Address::fromHorde(new Horde_Mail_Rfc822_Address("'[email protected]'"), false)->toHorde();
$this->assertEquals("'[email protected]'", $address->bare_address);
}

public function testNormalizedWithUpperCaseLetters() {
$address = Address::fromRaw(null, '[email protected]', true)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);

$address = Address::fromHorde(new Horde_Mail_Rfc822_Address('[email protected]'), true)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);
}

public function testUnnormalizedWithUpperCaseLetters() {
$address = Address::fromRaw(null, '[email protected]', false)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);

$address = Address::fromHorde(new Horde_Mail_Rfc822_Address('[email protected]'), false)->toHorde();
$this->assertEquals('[email protected]', $address->bare_address);
}
}