o 8@j$@sdZddlZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZedZeejZd d Zd d Zd dZe defddZdefddZdefddZdedefddZdedefddZdSdefdd Zd!efd"d#Z d$efd%d&Z!dedefd'd(Z"dedefd)d*Z#d+efd,d-Z$d.efd/d0Z%d$efd1d2Z&d3ed$efd4d5Z'd$efd6d7Z(d8efd9d:Z)d$efd;d<Z*d=efd>d?Z+d$efd@dAZ,dBefdCdDZ-d$efdEdFZ.dGefdHdIZ/d$efdJdKZ0dLefdMdNZ1dTdPdQZ2GdRdOdOZ3dS)Ua data_store.py All persistence lives here: premarket snapshots, intraday snapshots, trade records, and bot state (positions, watchlists, cooldowns). Design notes: - Uses simple JSON-lines files per day for premarket/intraday data so they can be analyzed later with pandas without a database dependency. - State files (positions.json, watchlist.json, cooldowns.json) are written atomically (write to tmp file, then os.replace) to avoid corruption if the process is killed mid-write. - A single filelock-style guard (using a lock file + fcntl) protects state writes in case of accidental concurrent processes, matching a lesson learned in prior systems where duplicate processes raced on shared state. N)datetimedate)Path)contextmanager) get_config) get_logger data_storecCs8td}t|dt|dt|dt|ddS)N data_storage premarket_dir intraday_dir trades_dir state_dir) premarketintradaytradesstate)rBASE_DIR)cfgrF/var/www/screener/trade/premarket_backup_2026-09-08_2010/data_store.py_dirs#s     rcCs"tD] }|jdddqdS)NTparentsexist_ok)rvaluesmkdir)drrr ensure_dirs-srcCs tSN)rtoday isoformatrrrr _today_str2s r! lock_pathc csz|jjdddt|d%}t|tjz dVWt|tjnt|tjwWddS1s6wYdS)NTrw)parentropenfcntlflockLOCK_EXLOCK_UN)r"lfrrr _file_lock6s ""r+pathcCsl|jjddd||jd}t|d}tj||dtdWdn1s)wYt ||dS)NTrz.tmpr#)indentdefault) r$r with_suffixsuffixr%jsondumpstrosreplace)r,datatmpfrrr_atomic_write_jsonAs  r:c Cs|s|Szt|d}t|WdWS1swYWdStjtfyG}ztd|d|d|WYd}~Sd}~ww)NrzFailed to read z: z. Returning default.)existsr%r2loadJSONDecodeErrorOSErrorlogwarning)r,r/r9errr _read_jsonIs (rCrecordcCsX|jjdddt|d}|tj|tddWddS1s%wYdS)NTra)r/ )r$rr%writer2dumpsr4)r,rDr9rrr _append_jsonlTs "rIsymbolcC:tdtd}t|d|}t||dS)Nr.jsonl timestamprJrr!rutcnowr rIrJrDr,rrrappend_premarket_snapshot^rRcandidates_20.json candidatescCs(tdtd|}t||dS)Nr_rr!r:)rUfilenamer,rrrwrite_premarket_candidatesdsrYtop_listcCs.tdd}t|d}t||dS)zDWrites the canonical top_stocks.json referenced by the whole system.rtop_stocks.json) updated_atstocksN)rrrPr r:)rZr,payloadrrrwrite_top_stocksisr_returncCs(tdd}t|dgi}|dgS)Nrr[r])rrCget)r,r7rrrread_top_stocksps rbcCrK)NrrLrMrOrQrrrappend_intraday_snapshotzrSrccCsrtdd}ttddt|i}dti|||<t||WddS1s2wYdS)zMirrors the latest intraday state for a symbol into intraday.json (a single mutable snapshot file, distinct from the historical .jsonl log), matching the project's requested intraday.json state file pattern.rz intraday.jsonzintraday.json.lockr\N)rr+rCrrPr r:)rJrDr,all_datarrrwrite_intraday_jsons   "retradecC$tdtd}t||dS)Nr _trades.jsonl)rr!rI)rfr,rrrappend_trade_recordrircCrg)Nrz _summary.jsonrW)rr,rrrwrite_trades_summaryrjrkc Cstdtd}|sgSg}t|d1}|D]%}|}|s$qz |t|Wqtjy@t d|YqwWd|S1sLwY|S)u Reads every trade record appended today via append_trade_record() — the complete, per-trade history for the day. [BUGFIX 2026-08-17] This is deliberately separate from PositionManager.positions, which is keyed by symbol and therefore only retains the MOST RECENT trade for any symbol traded more than once in a day. monitor.py's end-of-day summary previously built itself from positions.values() and silently undercounted days with repeated same-symbol round trips (verified: a session with 19 actual closed trades logged a summary of only 10 — one per unique symbol). Use this function instead for anything that needs a true count of the day's trades or an accurate total P/L. rrhr;z(Skipping malformed trade record line in N) rr!r<r%stripappendr2loadsr>r@rA)r,rr9linerrrload_today_tradess(    rpnamecCstd|S)Nr)r)rqrrr _state_pathrrcCs`ttdi}t}i}|D]\}}|ddvr |||<q|dd|r-|||<q|S)Npositions.jsonstatus)r%closing entry_time)rCrrr!itemsra startswith)rawrfilteredsymprrrload_positionss r positionscC@ttdttd|WddS1swYdS)Nzpositions.json.lockrtr+rrr:)rrrrsave_positions"rcCsttdggdS)Nwatchlist.json) premarket_20final_10rCrrrrrrload_watchlistsr watchlistcCttd|dS)Nrr:rr)rrrrsave_watchlistrcCttdiSNzcooldowns.jsonrrrrrload_cooldownsrsr cooldownscCrrr)rrrrsave_cooldownsrrcCr)Nintraday_health.jsonrrrrrload_health_statersr health_statecCr)Nzintraday_health.json.lockrr)rrrrsave_health_staterrcCr)Nsession_state.jsonrrrrrload_session_statersrrcCr)Nzsession_state.json.lockrr)rrrrsave_session_staterr SingletonLockcCs ttdS)uEnsures only one monitor.py instance runs at a time — prevents the duplicate-process-racing-the-same-account failure class.zmonitor.pid.lock)rrrrrrracquire_singleton_locks rc@s*eZdZdefddZddZddZdS) rr,cCs||_d|_dSr)r,_fh)selfr,rrr__init__s zSingletonLock.__init__cCst|jjjdddt|jd|_z t|jtjtjBWn t y(t dw|j t t |j|S)NTrr#z{Another monitor.py instance already holds the singleton lock. Refusing to start a second instance against the same account.)r,r$rr%rr&r'r(LOCK_NBBlockingIOError RuntimeErrorrGr4r5getpidflush)rrrr __enter__s  zSingletonLock.__enter__cCs(|jrt|jtj|jdSdSr)rr&r'r)close)rexc_typeexc_valexc_tbrrr__exit__szSingletonLock.__exit__N)__name__ __module__ __qualname__rrrrrrrrrs )rT)r`r)4__doc__r2r5r&timerrpathlibr contextlibr config_loaderr logger_setuprr@__file__resolver$rrrr!r+r:rCdictrIr4rRlistrYr_rbrcrerirkrprrrrrrrrrrrrrrrrrrsT         $