Various things

docs/publish.md

@@ -3161,16 +3161,20 @@ There are a few limitations to the API to prevent abuse and to keep the server h
 are configurable via the server side [rate limiting settings](config.md#rate-limiting). Most of these limits you won't run into,
 but just in case, let's list them all:
 
-| Limit                      | Description                                                                                                                                                              |
+| Limit                           | Description                                                                                                                                                                                                             |
-|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **Message length**         | Each message can be up to 4,096 bytes long. Longer messages are treated as [attachments](#attachments).                                                                  |
+| **Message length**              | Each message can be up to 4,096 bytes long. Longer messages are treated as [attachments](#attachments).                                                                                                                 |
-| **Requests**               | By default, the server is configured to allow 60 requests per visitor at once, and then refills the your allowed requests bucket at a rate of one request per 5 seconds. |
+| **Requests**                    | By default, the server is configured to allow 60 requests per visitor at once, and then refills the your allowed requests bucket at a rate of one request per 5 seconds.                                                |
-| **E-mails**                | By default, the server is configured to allow sending 16 e-mails per visitor at once, and then refills the your allowed e-mail bucket at a rate of one per hour.         |
+| **Daily messages**              | By default, the number of messages is governed by the request limits. This can be overridden. On ntfy.sh, the daily message limit is 1,000.                                                                             |
-| **Subscription limit**     | By default, the server allows each visitor to keep 30 connections to the server open.                                                                                    |
+| **E-mails**                     | By default, the server is configured to allow sending 16 e-mails per visitor at once, and then refills the your allowed e-mail bucket at a rate of one per hour. On ntfy.sh, the daily limit is 10.                     |
-| **Attachment size limit**  | By default, the server allows attachments up to 15 MB in size, up to 100 MB in total per visitor and up to 5 GB across all visitors.                                     |
+| **Subscription limit**          | By default, the server allows each visitor to keep 30 connections to the server open.                                                                                                                                   |
-| **Attachment expiry**      | By default, the server deletes attachments after 3 hours and thereby frees up space from the total visitor attachment limit.                                             |
+| **Attachment size limit**       | By default, the server allows attachments up to 15 MB in size, up to 100 MB in total per visitor and up to 5 GB across all visitors. On ntfy.sh, the attachment size limit is 5 MB, and the per-visitor total is 50 MB. |
-| **Attachment bandwidth**   | By default, the server allows 500 MB of GET/PUT/POST traffic for attachments per visitor in a 24 hour period. Traffic exceeding that is rejected.                        |
+| **Attachment expiry**           | By default, the server deletes attachments after 3 hours and thereby frees up space from the total visitor attachment limit.                                                                                            |
-| **Total number of topics** | By default, the server is configured to allow 15,000 topics. The ntfy.sh server has higher limits though.                                                                |
+| **Attachment bandwidth**        | By default, the server allows 500 MB of GET/PUT/POST traffic for attachments per visitor in a 24 hour period. Traffic exceeding that is rejected. On ntfy.sh, the daily bandwidth limit is 200 MB.                      |
+| **Total number of topics**      | By default, the server is configured to allow 15,000 topics. The ntfy.sh server has higher limits though.                                                                                                               |
+
+These limits can be changed on a per-user basis using [tiers](config.md#tiers). If [payments](config.md#payments) are enabled, a user tier can be changed by purchasing
+a higher tier. ntfy.sh offers multiple paid tiers, which allows for much hier limits than the ones listed above. 
 
 ## List of all parameters
 The following is a list of all parameters that can be passed when publishing a message. Parameter names are **case-insensitive**,

server/errors.go

@@ -39,7 +39,7 @@ func (e errHTTP) Context() log.Context {
 
 func (e errHTTP) Wrap(message string, args ...any) *errHTTP {
 	clone := e.clone()
-	clone.Message = fmt.Sprintf("%s, %s", clone.Message, fmt.Sprintf(message, args...))
+	clone.Message = fmt.Sprintf("%s; %s", clone.Message, fmt.Sprintf(message, args...))
 	return &clone
 }
 
@@ -115,9 +115,9 @@ var (
 	errHTTPEntityTooLargeAttachment                  = &errHTTP{41301, http.StatusRequestEntityTooLarge, "attachment too large, or bandwidth limit reached", "https://ntfy.sh/docs/publish/#limitations", nil}
 	errHTTPEntityTooLargeMatrixRequest               = &errHTTP{41302, http.StatusRequestEntityTooLarge, "Matrix request is larger than the max allowed length", "", nil}
 	errHTTPEntityTooLargeJSONBody                    = &errHTTP{41303, http.StatusRequestEntityTooLarge, "JSON body too large", "", nil}
-	errHTTPTooManyRequestsLimitRequests              = &errHTTP{42901, http.StatusTooManyRequests, "limit reached: too many requests, please be nice", "https://ntfy.sh/docs/publish/#limitations", nil}
+	errHTTPTooManyRequestsLimitRequests              = &errHTTP{42901, http.StatusTooManyRequests, "limit reached: too many requests", "https://ntfy.sh/docs/publish/#limitations", nil}
-	errHTTPTooManyRequestsLimitEmails                = &errHTTP{42902, http.StatusTooManyRequests, "limit reached: too many emails, please be nice", "https://ntfy.sh/docs/publish/#limitations", nil}
+	errHTTPTooManyRequestsLimitEmails                = &errHTTP{42902, http.StatusTooManyRequests, "limit reached: too many emails", "https://ntfy.sh/docs/publish/#limitations", nil}
-	errHTTPTooManyRequestsLimitSubscriptions         = &errHTTP{42903, http.StatusTooManyRequests, "limit reached: too many active subscriptions, please be nice", "https://ntfy.sh/docs/publish/#limitations", nil}
+	errHTTPTooManyRequestsLimitSubscriptions         = &errHTTP{42903, http.StatusTooManyRequests, "limit reached: too many active subscriptions", "https://ntfy.sh/docs/publish/#limitations", nil}
 	errHTTPTooManyRequestsLimitTotalTopics           = &errHTTP{42904, http.StatusTooManyRequests, "limit reached: the total number of topics on the server has been reached, please contact the admin", "https://ntfy.sh/docs/publish/#limitations", nil}
 	errHTTPTooManyRequestsLimitAttachmentBandwidth   = &errHTTP{42905, http.StatusTooManyRequests, "limit reached: daily bandwidth reached", "https://ntfy.sh/docs/publish/#limitations", nil}
 	errHTTPTooManyRequestsLimitAccountCreation       = &errHTTP{42906, http.StatusTooManyRequests, "limit reached: too many accounts created", "https://ntfy.sh/docs/publish/#limitations", nil} // FIXME document limit

server/log.go

@@ -31,7 +31,8 @@ const (
 )
 
 var (
-	normalErrorCodes = []int{http.StatusNotFound, http.StatusBadRequest, http.StatusTooManyRequests, http.StatusUnauthorized, http.StatusInsufficientStorage}
+	normalErrorCodes       = []int{http.StatusNotFound, http.StatusBadRequest, http.StatusTooManyRequests, http.StatusUnauthorized, http.StatusInsufficientStorage}
+	rateLimitingErrorCodes = []int{http.StatusTooManyRequests, http.StatusRequestEntityTooLarge}
 )
 
 // logr creates a new log event with HTTP request fields

server/server.go

@@ -319,6 +319,7 @@ func (s *Server) handleError(w http.ResponseWriter, r *http.Request, v *visitor,
 	if !ok {
 		httpErr = errHTTPInternalError
 	}
+	isRateLimiting := util.Contains(rateLimitingErrorCodes, httpErr.HTTPCode)
 	isNormalError := strings.Contains(err.Error(), "i/o timeout") || util.Contains(normalErrorCodes, httpErr.HTTPCode)
 	ev := logvr(v, r).Err(err)
 	if websocket.IsWebSocketUpgrade(r) {
@@ -335,6 +336,12 @@ func (s *Server) handleError(w http.ResponseWriter, r *http.Request, v *visitor,
 	} else {
 		ev.Info("Connection closed with HTTP %d (ntfy error %d)", httpErr.HTTPCode, httpErr.Code)
 	}
+	if isRateLimiting && s.config.StripeSecretKey != "" {
+		u := v.User()
+		if u == nil || u.Tier == nil {
+			httpErr = httpErr.Wrap("increase your limits with a paid plan, see %s", s.config.BaseURL)
+		}
+	}
 	w.Header().Set("Content-Type", "application/json")
 	w.Header().Set("Access-Control-Allow-Origin", s.config.AccessControlAllowOrigin) // CORS, allow cross-origin requests
 	w.WriteHeader(httpErr.HTTPCode)
@@ -509,7 +516,10 @@ func (s *Server) handleFile(w http.ResponseWriter, r *http.Request, v *visitor)
 	file := filepath.Join(s.config.AttachmentCacheDir, messageID)
 	stat, err := os.Stat(file)
 	if err != nil {
-		return errHTTPNotFound
+		return errHTTPNotFound.Fields(log.Context{
+			"message_id":    messageID,
+			"error_context": "filesystem",
+		})
 	}
 	w.Header().Set("Access-Control-Allow-Origin", s.config.AccessControlAllowOrigin) // CORS, allow cross-origin requests
 	w.Header().Set("Content-Length", fmt.Sprintf("%d", stat.Size()))
@@ -530,7 +540,10 @@ func (s *Server) handleFile(w http.ResponseWriter, r *http.Request, v *visitor)
 			}, s.config.CacheBatchTimeout, 100*time.Millisecond, 300*time.Millisecond, 600*time.Millisecond)
 		}
 		if err != nil {
-			return errHTTPNotFound
+			return errHTTPNotFound.Fields(log.Context{
+				"message_id":    messageID,
+				"error_context": "message_cache",
+			})
 		}
 	} else if err != nil {
 		return err
@@ -546,7 +559,7 @@ func (s *Server) handleFile(w http.ResponseWriter, r *http.Request, v *visitor)
 		bandwidthVisitor = s.visitor(m.Sender, nil)
 	}
 	if !bandwidthVisitor.BandwidthAllowed(stat.Size()) {
-		return errHTTPTooManyRequestsLimitAttachmentBandwidth
+		return errHTTPTooManyRequestsLimitAttachmentBandwidth.With(m)
 	}
 	// Actually send file
 	f, err := os.Open(file)
@@ -866,13 +879,17 @@ func (s *Server) handleBodyAsAttachment(r *http.Request, v *visitor, m *message,
 	}
 	attachmentExpiry := time.Now().Add(vinfo.Limits.AttachmentExpiryDuration).Unix()
 	if m.Time > attachmentExpiry {
-		return errHTTPBadRequestAttachmentsExpiryBeforeDelivery
+		return errHTTPBadRequestAttachmentsExpiryBeforeDelivery.With(m)
 	}
 	contentLengthStr := r.Header.Get("Content-Length")
 	if contentLengthStr != "" { // Early "do-not-trust" check, hard limit see below
 		contentLength, err := strconv.ParseInt(contentLengthStr, 10, 64)
 		if err == nil && (contentLength > vinfo.Stats.AttachmentTotalSizeRemaining || contentLength > vinfo.Limits.AttachmentFileSizeLimit) {
-			return errHTTPEntityTooLargeAttachment
+			return errHTTPEntityTooLargeAttachment.With(m).Fields(log.Context{
+				"message_content_length":          contentLength,
+				"attachment_total_size_remaining": vinfo.Stats.AttachmentTotalSizeRemaining,
+				"attachment_file_size_limit":      vinfo.Limits.AttachmentFileSizeLimit,
+			})
 		}
 	}
 	if m.Attachment == nil {