gomail: could not send email 1: gomail: invalid address "": mail: no address

[somera]

Gitea Version

1.16.0

Git Version

2.25.1

Operating System

Ubuntu 20.04.3

How are you running Gitea?

Precompiled gitea-1.16.0-linux-amd64

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

During gitea restart I saw this

2022/02/05 12:06:54 ...ces/mailer/mailer.go:359:func1() [E] Failed to send emails []:  - gomail: could not send email 1: gomail: invalid address "": mail: no address
        /source/services/mailer/mailer.go:359 (0x14bb771)
        /source/modules/queue/workerpool.go:369 (0x149c85a)
        /source/modules/queue/workerpool.go:261 (0x149bc56)
        /usr/local/go/src/runtime/asm_amd64.s:1581 (0x471520)

errors 1848 times in one row in the log.

My questions:

• What kind of email?
• For what action?
• To which email? User's email?

Screenshots

No response

[zeripath]

This error implies that something tried to send an email to a user with an empty email address. As far as the mailer is concerned the mail was to no-one and had no information at all it. That's why there's no information.

The only thing I can think of is that there is a difference in format of the mail messages on the queue as compared to previously - but I can't see any convincing evidence of that in the source code.

(When you shutdown gitea before upgrading it's generally recommended that you flush your queues so that on startup there isn't anything the queue in case there are version changes. )

[somera]

This error implies that something tried to send an email to a user with an empty email address. As far as the mailer is concerned the mail was to no-one and had no information at all it. That's why there's no information.

But 1848 times in less than 1 second?

My user has defined email address.

And before the restart I tested my new sync mirror problems (#18607). Maybe it has something to do with this?

The only thing I can think of is that there is a difference in format of the mail messages on the queue as compared to previously - but I can't see any convincing evidence of that in the source code.

(When you shutdown gitea before upgrading it's generally recommended that you flush your queues so that on startup there isn't anything the queue in case there are version changes. )

How can I flush the queue?

[zeripath]

This error implies that something tried to send an email to a user with an empty email address. As far as the mailer is concerned the mail was to no-one and had no information at all it. That's why there's no information.

But 1848 times in less than 1 second?

Something has run services/mailer/SendAsync(msg *Message) asking it to send email to no-one. (or there are some weird broken entries in the queue.)

My user has defined email address.

That may well be - but something has tried to send these emails to something without an email address... There might be more information higher up in the logs.

And before the restart I tested my new sync mirror problems (#18607). Maybe it has something to do with this?

Possibly. It raises the likelihood that your queues are completely corrupt. What is your queue configuration? Do you have multiple Gitea heads?

You might be better off clearing them out and starting again.

The only thing I can think of is that there is a difference in format of the mail messages on the queue as compared to previously - but I can't see any convincing evidence of that in the source code.
(When you shutdown gitea before upgrading it's generally recommended that you flush your queues so that on startup there isn't anything the queue in case there are version changes. )

How can I flush the queue?

./gitea manager flush-queues

Or flush each queue manually from the monitoring page.

[somera]

And before the restart I tested my new sync mirror problems (#18607). Maybe it has something to do with this?

Possibly. It raises the likelihood that your queues are completely corrupt. What is your queue configuration? Do you have multiple Gitea heads?

I don't have any special [queue.*] configuration. But now I added:

[queue.mirror]
LENGTH = 20000

You might be better off clearing them out and starting again.

The only thing I can think of is that there is a difference in format of the mail messages on the queue as compared to previously - but I can't see any convincing evidence of that in the source code.
(When you shutdown gitea before upgrading it's generally recommended that you flush your queues so that on startup there isn't anything the queue in case there are version changes. )

How can I flush the queue?

./gitea manager flush-queues

Thx.

Should this be done on every Gitea Update?

Where is the queue stored? Memcached? DB?

Or flush each queue manually from the monitoring page.

[zeripath]

How are you running gitea? Are you running multiple Giteas in a cluster?

You can check your queue configuration on monitor, review any queue and it will tell you what the inferred configuration is for that queue.

On disk queues will store their data by default in DATA_DIR/queues/common

---

In most cases the queue will be a persistable-channel (meaning that there will generally be two or possibly three queues listed)

A persistable-channel queue are two queues tied together:

• An in memory channel queue
• An level queue on disk

Whilst Gitea is running the in memory channel is used. On shutdown any unprocessed entries are persisted down to the disk queue. On startup the disk queue is emptied with new entries being added and processed from the memory channel.

A level queue just has the on disk version.

---

If you're running multiple gitea heads you should use a redis queue.

[somera]

How are you running gitea? Are you running multiple Giteas in a cluster?

No. Single instance. Prevompiled gitea--linux-amd64

You can check your queue configuration on monitor, review any queue and it will tell you what the inferred configuration is for that queue.

{
  "Name": "mirror",
  "DataDir": "/var/lib/gitea/data/queues/common",
  "BatchLength": 20,
  "QueueLength": 20000,
  "Timeout": 90000000000,
  "MaxAttempts": 10,
  "Workers": 0,
  "MaxWorkers": 10,
  "BlockTimeout": 1000000000,
  "BoostTimeout": 300000000000,
  "BoostWorkers": 1
}

{
  "QueueLength": 20000,
  "BatchLength": 20,
  "BlockTimeout": 1000000000,
  "BoostTimeout": 300000000000,
  "BoostWorkers": 1,
  "MaxWorkers": 10,
  "Workers": 0,
  "Name": "mirror-channel"
}

On disk queues will store their data by default in DATA_DIR/queues/common

In most cases the queue will be a persistable-channel (meaning that there will generally be two or possibly three queues listed)

A persistable-channel queue are two queues tied together:

• An in memory channel queue
• An level queue on disk

Whilst Gitea is running the in memory channel is used. On shutdown any unprocessed entries are persisted down to the disk queue. On startup the disk queue is emptied with new entries being added and processed from the memory channel.

A level queue just has the on disk version.

If you're running multiple gitea heads you should use a redis queue.

Thx.

[somera]

@zeripath should this be closen? I didn't see this error anymore.

[zeripath]

If it's no longer happening then we can close