200 lines
5.6 KiB
Markdown
200 lines
5.6 KiB
Markdown
# Network Logging Minimal Example
|
|
|
|
Use this reference when an application should send logs over TCP to a local receiver and you want a complete, working baseline.
|
|
|
|
This page shows how the pieces fit together end to end:
|
|
|
|
- application code logs with named loggers
|
|
- startup applies one declarative `LOGGING` config
|
|
- `SocketHandler` sends records to a receiver
|
|
- receiver uses `socketserver` and local logging config for final routing
|
|
|
|
Source docs to keep nearby:
|
|
|
|
- [Sending and receiving logging events across a network](https://docs.python.org/3/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network)
|
|
- [`SocketHandler`](https://docs.python.org/3/library/logging.handlers.html#sockethandler)
|
|
- [`socketserver`](https://docs.python.org/3/library/socketserver.html)
|
|
- [`logging.makeLogRecord`](https://docs.python.org/3/library/logging.html#logging.makeLogRecord)
|
|
|
|
## Minimal Topology
|
|
|
|
```text
|
|
app module -> logger -> SocketHandler -> TCP receiver -> local handlers
|
|
```
|
|
|
|
## 1) Client Logging Config (Declarative)
|
|
|
|
```python title="logging_config.py"
|
|
import logging.config
|
|
|
|
LOGGING = {
|
|
"version": 1,
|
|
"disable_existing_loggers": False,
|
|
"formatters": {
|
|
"console": {
|
|
"format": "%(asctime)s %(levelname)s %(name)s %(message)s",
|
|
"datefmt": "%Y-%m-%d %H:%M:%S",
|
|
}
|
|
},
|
|
"handlers": {
|
|
"network": {
|
|
"class": "logging.handlers.SocketHandler",
|
|
"host": "127.0.0.1",
|
|
"port": 9020,
|
|
},
|
|
"console": {
|
|
"class": "logging.StreamHandler",
|
|
"formatter": "console",
|
|
"level": "INFO",
|
|
"stream": "ext://sys.stdout",
|
|
},
|
|
},
|
|
"root": {
|
|
"level": "INFO",
|
|
"handlers": ["network", "console"],
|
|
},
|
|
}
|
|
|
|
|
|
def configure_logging() -> None:
|
|
logging.config.dictConfig(LOGGING)
|
|
```
|
|
|
|
```python title="feature.py"
|
|
import logging
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def process_order(order_id: str) -> None:
|
|
logger.info("Processing order %s", order_id)
|
|
```
|
|
|
|
```python title="main.py"
|
|
from feature import process_order
|
|
from logging_config import configure_logging
|
|
|
|
|
|
def main() -> None:
|
|
configure_logging()
|
|
process_order("A-42")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
## 2) Receiver Logging Config (Declarative)
|
|
|
|
```python title="receiver_logging_config.py"
|
|
import logging.config
|
|
|
|
LOGGING = {
|
|
"version": 1,
|
|
"disable_existing_loggers": False,
|
|
"formatters": {
|
|
"console": {
|
|
"format": "%(asctime)s %(levelname)s %(name)s %(message)s",
|
|
"datefmt": "%Y-%m-%d %H:%M:%S",
|
|
}
|
|
},
|
|
"handlers": {
|
|
"console": {
|
|
"class": "logging.StreamHandler",
|
|
"formatter": "console",
|
|
"stream": "ext://sys.stdout",
|
|
}
|
|
},
|
|
"root": {"level": "INFO", "handlers": ["console"]},
|
|
}
|
|
|
|
|
|
def configure_receiver_logging() -> None:
|
|
logging.config.dictConfig(LOGGING)
|
|
```
|
|
|
|
## 3) Cookbook Receiver (`socketserver`) Implementation
|
|
|
|
This receiver follows the same structure as the Python logging cookbook example.
|
|
|
|
`SocketHandler` sends:
|
|
|
|
- a 4-byte big-endian length prefix
|
|
- a pickle payload containing a `LogRecord` dictionary
|
|
|
|
```python title="log_receiver.py"
|
|
import logging
|
|
import pickle
|
|
import socketserver
|
|
import struct
|
|
|
|
from receiver_logging_config import configure_receiver_logging
|
|
|
|
|
|
class LogRecordStreamHandler(socketserver.StreamRequestHandler):
|
|
def handle(self) -> None:
|
|
while True:
|
|
chunk = self.connection.recv(4)
|
|
if len(chunk) < 4:
|
|
break
|
|
|
|
payload_len = struct.unpack(">L", chunk)[0]
|
|
payload = self.connection.recv(payload_len)
|
|
while len(payload) < payload_len:
|
|
payload = payload + self.connection.recv(payload_len - len(payload))
|
|
|
|
try:
|
|
record_dict = pickle.loads(payload)
|
|
record = logging.makeLogRecord(record_dict)
|
|
except Exception:
|
|
logging.getLogger(__name__).exception("Dropped malformed log record")
|
|
continue
|
|
|
|
self.handle_log_record(record)
|
|
|
|
def handle_log_record(self, record: logging.LogRecord) -> None:
|
|
logger = logging.getLogger(record.name)
|
|
if logger.isEnabledFor(record.levelno):
|
|
logger.handle(record)
|
|
|
|
|
|
class LogRecordSocketReceiver(socketserver.ThreadingTCPServer):
|
|
allow_reuse_address = True
|
|
|
|
|
|
def main() -> None:
|
|
configure_receiver_logging()
|
|
with LogRecordSocketReceiver(("127.0.0.1", 9020), LogRecordStreamHandler) as server:
|
|
logging.getLogger(__name__).info("Receiver listening on 127.0.0.1:9020")
|
|
server.serve_forever()
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
## 4) How It Fits Together In Practice
|
|
|
|
1. Start `log_receiver.py`.
|
|
2. Start `main.py` from the client app.
|
|
3. Client logs go to console and TCP.
|
|
4. Receiver reconstructs records and emits them through its own handlers.
|
|
|
|
This split keeps app emission and receiver routing independent while still being fully runnable.
|
|
|
|
## Important Security Note
|
|
|
|
`SocketHandler` uses pickle serialization. Treat this as trusted-network-only transport.
|
|
|
|
- Bind receiver to localhost or a trusted private network.
|
|
- Do not expose this receiver to untrusted clients.
|
|
- For hostile boundaries, use JSON/TLS with authenticated ingestion instead of raw pickle.
|
|
|
|
## Review Checklist
|
|
|
|
1. Is there one `LOGGING` dict per process role (client and receiver)?
|
|
2. Is `dictConfig` called once at each process startup?
|
|
3. Does the receiver decode length-prefixed payloads correctly?
|
|
4. Do modules only use `logging.getLogger(__name__)`?
|
|
5. Is the receiver endpoint protected by trust boundaries?
|