Merge remote-tracking branch 'refs/remotes/origin/master' into module…

…-execution-require-config-to-invoke

.github/pull_request_template.md

@@ -19,6 +19,15 @@ What's the context for the changes? Are there any
 Why did you choose to make these changes? Were there any trade-offs you had to consider?
 
 
+### 🔎 New Bid Adapter Checklist
+- [ ] verify email contact works
+- [ ] NO fully dynamic hosts
+- [ ] geographic host parameters are NOT required
+- [ ] NO direct use of HTTP is prohibited - *implement an existing Bidder interface that will do all the job*
+- [ ] if the ORTB is just forwarded to the endpoint, use the generic adapter - *define the new adapter as the alias of the generic adapter*
+- [ ] cover an adapter configuration with an integration test
+
+
 ### 🧪 Test plan
 
 How do you know the changes are safe to ship to production?

docs/admin-endpoints.md

@@ -0,0 +1,209 @@
+# Admin enpoints
+
+Prebid Server Java offers a set of admin endpoints for managing and monitoring the server's health, configurations, and
+metrics. Below is a detailed description of each endpoint, including HTTP methods, paths, parameters, and responses.
+
+## General settings
+
+Each endpoint can be either enabled or disabled by changing `admin-endpoints.<endoint name>.enabled` toggle. Defaults to
+`false`.
+
+Each endpoint can be configured to serve either on application port (configured via `server.http.port` setting) or
+admin port (configured via `admin.port` setting) by changing `admin-endpoints.<endpoint name>.on-application-port`
+setting.
+By default, all admin endpoints reside on admin port.
+
+Each endpoint can be configured to serve on a certain path by setting `admin-endpoints.<endpoint name>.path`.
+
+Each endpoint can be configured to either require basic authorization or not by changing
+`admin-endpoints.<endpoint name>.protected` setting,
+defaults to `true`. Allowed credentials are globally configured for all admin endpoints with
+`admin-endpoints.credentials.<username>`
+setting.
+
+## Endpoints
+
+1. Version info
+
+- Name: version
+- Endpoint: Configured via `admin-endpoints.version.path` setting
+- Methods:
+    - `GET`:
+        - Description: Returns the version information for the Prebid Server Java instance.
+        - Parameters: None
+        - Responses:
+            - 200 OK: JSON containing version details
+            ```json
+            {
+                "version": "x.x.x",
+                "revision": "commit-hash"
+            }
+           ```
+
+2. Currency rates
+
+- Name: currency-rates
+- Methods:
+    - `GET`:
+        - Description: Returns the latest information about currency rates used by server instance.
+        - Parameters: None
+        - Responses:
+            - 200 OK: JSON containing version details
+            ```json
+            {
+                "active": "true",
+                "source": "http://currency-source"
+                "fetchingIntervalNs": 200,
+                "lastUpdated": "02/01/2018 - 13:45:30 UTC"
+                ... Rates ...
+            }
+           ```
+
+3. Cache notification endpoint
+
+- Name: storedrequest
+- Methods:
+    - `POST`:
+        - Description: Updates stored requests/imps data stored in server instance cache.
+        - Parameters:
+            - body:
+              ```json
+              {
+                  "requests": {
+                      "<request-name-1>": "<body-1>",
+                      ... Requests data ...
+                  },
+                  "imps": {
+                      "<imp-name-1>": "<body-1>",
+                      ... Imps data ...
+                  }
+              }
+              ``` 
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+            - 405 METHOD NOT ALLOWED
+    - `DELETE`:
+        - Description: Invalidates stored requests/imps data stored in server instance cache.
+        - Parameters:
+            - body:
+              ```json
+              {
+                  "requests": ["<request-name-1>", ... Request names ...],
+                  "imps": ["<imp-name-1>", ... Imp names ...]
+              }
+              ``` 
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+            - 405 METHOD NOT ALLOWED
+
+4. Amp cache notification endpoint
+
+- Name: storedrequest-amp
+- Methods:
+    - `POST`:
+        - Description: Updates stored requests/imps data for amp, stored in server instance cache.
+        - Parameters:
+            - body:
+              ```json
+              {
+                  "requests": {
+                      "<request-name-1>": "<body-1>",
+                      ... Requests data ...
+                  },
+                  "imps": {
+                      "<imp-name-1>": "<body-1>",
+                      ... Imps data ...
+                  }
+              }
+              ``` 
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+            - 405 METHOD NOT ALLOWED
+    - `DELETE`:
+        - Description: Invalidates stored requests/imps data for amp, stored in server instance cache.
+        - Parameters:
+            - body:
+              ```json
+              {
+                  "requests": ["<request-name-1>", ... Request names ...],
+                  "imps": ["<imp-name-1>", ... Imp names ...]
+              }
+              ``` 
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+            - 405 METHOD NOT ALLOWED
+
+5. Account cache notification endpoint
+
+- Name: cache-invalidation
+- Methods:
+    - any:
+        - Description: Invalidates cached data for a provided account in server instance cache.
+        - Parameters:
+            - `account`: Account id.
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+
+
+6. Http interaction logging endpoint
+
+- Name: logging-httpinteraction
+- Methods:
+    - any:
+        - Description: Changes request logging specification in server instance.
+        - Parameters:
+            - `endpoint`: Endpoint. Should be either: `auction` or `amp`.
+            - `statusCode`: Status code for logging spec.
+            - `account`: Account id.
+            - `bidder`: Bidder code.
+            - `limit`: Limit of requests for specification to be valid.
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+- Additional settings:
+    - `logging.http-interaction.max-limit` - max limit for logging specification limit.
+
+7. Logging level control endpoint
+
+- Name: logging-changelevel
+- Methods:
+    - any:
+        - Description: Changes request logging level for specified amount of time in server instance.
+        - Parameters:
+            - `level`: Logging level. Should be one of: `all`, `trace`, `debug`, `info`, `warn`, `error`, `off`.
+            - `duration`: Duration of logging level (in millis) before reset to original one.
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+- Additional settings:
+    - `logging.change-level.max-duration-ms` - max duration of changed logger level.
+
+8. Tracer log endpoint
+
+- Name: tracelog
+- Methods:
+    - any:
+        - Description: Adds trace logging specification for specified amount of time in server instance.
+        - Parameters:
+            - `account`: Account id.
+            - `bidderCode`: Bidder code.
+            - `level`: Log level. Should be one of: `info`, `warn`, `trace`, `error`, `fatal`, `debug`.
+            - `duration`: Duration of logging specification (in seconds).
+        - Responses:
+            - 200 OK
+            - 400 BAD REQUEST
+
+9. Collected metrics endpoint
+
+- Name: collected-metrics
+- Methods:
+    - any:
+        - Description: Adds trace logging specification for specified amount of time in server instance.
+        - Parameters: None
+        - Responses:
+            - 200 OK: JSON containing metrics data.

src/main/java/org/prebid/server/analytics/reporter/agma/AgmaAnalyticsReporter.java

@@ -16,6 +16,7 @@
 import io.vertx.core.Vertx;
 import io.vertx.core.http.HttpHeaders;
 import io.vertx.core.http.HttpMethod;
+import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.lang3.tuple.Pair;
 import org.prebid.server.analytics.AnalyticsReporter;
 import org.prebid.server.analytics.model.AmpEvent;
@@ -146,11 +147,7 @@ public <T> Future<Void> processEvent(T event) {
 
         final String eventString = jacksonMapper.encodeToString(agmaEvent);
         buffer.put(eventString, eventString.length());
-        final List<String> toFlush = buffer.pollToFlush();
-        if (!toFlush.isEmpty()) {
-            sendEvents(toFlush);
-        }
-
+        sendEvents(buffer.pollToFlush());
         return Future.succeededFuture();
     }
 
@@ -200,10 +197,15 @@ private static String getPublisherId(BidRequest bidRequest) {
             return null;
         }
 
-        return publisherId;
+        return StringUtils.isNotBlank(appSiteId)
+                ? String.format("%s_%s", StringUtils.defaultString(publisherId), appSiteId)
+                : publisherId;
     }
 
     private void sendEvents(List<String> events) {
+        if (events.isEmpty()) {
+            return;
+        }
         final String payload = preparePayload(events);
         final Future<HttpClientResponse> responseFuture = compressToGzip
                 ? httpClient.request(HttpMethod.POST, url, headers, gzip(payload), httpTimeoutMs)

src/main/java/org/prebid/server/auction/PriceGranularity.java

@@ -72,6 +72,12 @@ public static PriceGranularity createFromString(String stringPriceGranularity) {
         }
     }
 
+    public static PriceGranularity createFromStringOrDefault(String stringPriceGranularity) {
+        return isValidStringPriceGranularityType(stringPriceGranularity)
+                ? STRING_TO_CUSTOM_PRICE_GRANULARITY.get(PriceGranularityType.valueOf(stringPriceGranularity))
+                : PriceGranularity.DEFAULT;
+    }
+
     /**
      * Returns list of {@link ExtGranularityRange}s.
      */

src/main/java/org/prebid/server/auction/requestfactory/AmpRequestFactory.java

@@ -52,6 +52,7 @@
 import org.prebid.server.proto.openrtb.ext.request.ExtStoredRequest;
 import org.prebid.server.proto.openrtb.ext.request.ExtUser;
 import org.prebid.server.settings.model.Account;
+import org.prebid.server.settings.model.AccountAuctionConfig;
 import org.prebid.server.util.HttpUtil;
 
 import java.util.ArrayList;
@@ -60,6 +61,7 @@
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import java.util.Optional;
 import java.util.stream.Collectors;
 
 public class AmpRequestFactory {
@@ -407,7 +409,7 @@ private Future<BidRequest> updateBidRequest(AuctionContext auctionContext) {
                 .map(ortbVersionConversionManager::convertToAuctionSupportedVersion)
                 .map(bidRequest -> gppService.updateBidRequest(bidRequest, auctionContext))
                 .map(bidRequest -> validateStoredBidRequest(storedRequestId, bidRequest))
-                .map(this::fillExplicitParameters)
+                .map(bidRequest -> fillExplicitParameters(bidRequest, account))
                 .map(bidRequest -> overrideParameters(bidRequest, httpRequest, auctionContext.getPrebidErrors()))
                 .map(bidRequest -> paramsResolver.resolve(bidRequest, auctionContext, ENDPOINT, true))
                 .map(bidRequest -> ortb2RequestFactory.removeEmptyEids(bidRequest, auctionContext.getDebugWarnings()))
@@ -459,7 +461,7 @@ private static BidRequest validateStoredBidRequest(String tagId, BidRequest bidR
      * - Sets {@link BidRequest}.test = 1 if it was passed in {@link RoutingContext}
      * - Updates {@link BidRequest}.ext.prebid.amp.data with all query parameters
      */
-    private BidRequest fillExplicitParameters(BidRequest bidRequest) {
+    private BidRequest fillExplicitParameters(BidRequest bidRequest, Account account) {
         final List<Imp> imps = bidRequest.getImp();
         // Force HTTPS as AMP requires it, but pubs can forget to set it.
         final Imp imp = imps.getFirst();
@@ -496,6 +498,7 @@ private BidRequest fillExplicitParameters(BidRequest bidRequest) {
                     .imp(setSecure ? Collections.singletonList(imps.getFirst().toBuilder().secure(1).build()) : imps)
                     .ext(extRequest(
                             bidRequest,
+                            account,
                             setDefaultTargeting,
                             setDefaultCache))
                     .build();
@@ -692,6 +695,7 @@ private static List<Format> parseMultiSizeParam(String ms) {
      * Creates updated bidrequest.ext {@link ObjectNode}.
      */
     private ExtRequest extRequest(BidRequest bidRequest,
+                                  Account account,
                                   boolean setDefaultTargeting,
                                   boolean setDefaultCache) {
 
@@ -704,7 +708,7 @@ private ExtRequest extRequest(BidRequest bidRequest,
                     : ExtRequestPrebid.builder();
 
             if (setDefaultTargeting) {
-                prebidBuilder.targeting(createTargetingWithDefaults(prebid));
+                prebidBuilder.targeting(createTargetingWithDefaults(prebid, account));
             }
             if (setDefaultCache) {
                 prebidBuilder.cache(ExtRequestPrebidCache.of(ExtRequestPrebidCacheBids.of(null, null),
@@ -727,15 +731,14 @@ private ExtRequest extRequest(BidRequest bidRequest,
      * Creates updated with default values bidrequest.ext.targeting {@link ExtRequestTargeting} if at least one of it's
      * child properties is missed or entire targeting does not exist.
      */
-    private ExtRequestTargeting createTargetingWithDefaults(ExtRequestPrebid prebid) {
+    private ExtRequestTargeting createTargetingWithDefaults(ExtRequestPrebid prebid, Account account) {
         final ExtRequestTargeting targeting = prebid != null ? prebid.getTargeting() : null;
         final boolean isTargetingNull = targeting == null;
 
         final JsonNode priceGranularityNode = isTargetingNull ? null : targeting.getPricegranularity();
         final boolean isPriceGranularityNull = priceGranularityNode == null || priceGranularityNode.isNull();
-        final JsonNode outgoingPriceGranularityNode
-                = isPriceGranularityNull
-                ? mapper.mapper().valueToTree(ExtPriceGranularity.from(PriceGranularity.DEFAULT))
+        final JsonNode outgoingPriceGranularityNode = isPriceGranularityNull
+                ? mapper.mapper().valueToTree(ExtPriceGranularity.from(getDefaultPriceGranularity(account)))
                 : priceGranularityNode;
 
         final ExtMediaTypePriceGranularity mediaTypePriceGranularity = isTargetingNull
@@ -759,6 +762,14 @@ private ExtRequestTargeting createTargetingWithDefaults(ExtRequestPrebid prebid)
                 .build();
     }
 
+    private static PriceGranularity getDefaultPriceGranularity(Account account) {
+        return Optional.ofNullable(account)
+                .map(Account::getAuction)
+                .map(AccountAuctionConfig::getPriceGranularity)
+                .map(PriceGranularity::createFromStringOrDefault)
+                .orElse(PriceGranularity.DEFAULT);
+    }
+
     @Value(staticConstructor = "of")
     private static class GppSidExtraction {