o ַjK@sRdZddlZddlmZmZmZddlmZddlmZddl m Z ddl m Z ddl mZddlZddlZed Zd efd d Zd efd dZ  d,dedededed ef ddZded efddZ   d-dedededededed efddZed kreZeD]#Zeed!d"d#ed$d%d&ed'd(d)d*ed'd+d)qdSdS).u= premarket_scanner.py Runs at ~09:00 ET. Scans the tradable universe ($5-$15, liquid enough) and produces a ranked list of candidates using scorer.py. Called by monitor.py, but also runnable standalone for testing: python premarket_scanner.py It does NOT buy anything — this module only discovers and ranks. N)datetime timedeltatimezone) get_config) get_logger) get_client)score_premarket_candidate)atrpremarket_scannerreturnc CsJg}|D]}||jt|jt|jt|jt|jt|jdq|S)N)tohlcv)append timestampfloatopenhighlowclosevolume)barsoutbrpremarket_scanner.py_bars_to_dictss rcstd}|dr|dS|}t|dgfdd|D}t|}tj|dd}td|t|d |d |S) a Pulls active, tradable US equities and applies: 1. the include/exclude_symbols lists from config 2. ETF/fund/trust exclusion (symbol_filters.is_etf_or_fund) 3. leveraged/inverse ("multiplier") product exclusion 4. company-name syllable-count cap Alpaca's asset_class=US_EQUITY includes ETFs/ETNs, so filters 2-4 are done on the asset's `name` field via symbol_filters.py. universeinclude_only_symbolsexclude_symbolscs&g|]}t|ddr|jvr|qS)tradableF)getattrsymbol).0aexcludedrr 7s&z(get_universe_symbols..T)log_rejectionszC[PREMARKET] Symbol filters (ETF/leveraged/name-complexity) removed z of candidates) rgetget_tradable_assetssetlensymbol_filtersfilter_symbol_listloginfo)clientcfgassets candidatesbefore survivorsrr(rget_universe_symbols%s   r;symbols baseline_outprev_day_high_outprev_close_outc Cs^td}td}g}||}|s|S|D]\} } z| j} | j} | dur*Wqt| j} |d| kr=|dks@nWq| rGt| jnd}|rU||ddkrUWqt| d d}|durw|jrwt|j}||dkroWq|durw||| <|dur|dur|j rt|j || <|dur|dur|j rt|j || <| | Wqt t fyYqw|S) a Cheap first pass using snapshot data to cut the universe down to a manageable set before pulling minute bars for full scoring. Filters on price band and a basic volume floor. baseline_out: [Phase 5 -- 2026-09-08, BUGFIXED 2026-09-15] if a dict is passed, this function populates it with {symbol: real_prior_day_ volume} as a side effect (mutated in place, return value unchanged). Originally sourced from `snap.daily_bar.volume` -- that was itself a fix for an older bug (every RVOL calculation comparing against the same flat universe.min_avg_daily_volume constant regardless of a symbol's real normal volume) but introduced a worse one: `daily_bar` is the CURRENT session's bar, which Alpaca already starts accumulating from 4:00am ET extended hours -- so by the ~09:00 scan time it's not a historical baseline at all, it's just today's premarket volume so far. A few minutes later scan() sums those same premarket bars into total_volume, so RVOL ended up comparing today's premarket volume to itself (~1.0x for nearly everything, real catalyst or not), silently killing the scanner's main volume-interest signal from 2026-09-08 onward. Now sourced from `snap.previous_daily_bar. volume` instead (the prior COMPLETE session's real volume) -- same fix pattern already used for prev_day_high_out below. Omit baseline_out (the default) to reproduce this function's exact pre-Phase-5 behavior -- callers that don't pass it are completely unaffected. prev_day_high_out: [FEATURE 2026-09-11, attribute name BUGFIXED 2026-09-15] same pattern as baseline_out above, for fast_prediction_engine.py's resistance-context layer. Populated with {symbol: previous session's high} from Alpaca's snapshot -- reads snap.previous_daily_bar (unambiguous, always "the prior completed session") rather than snap.daily_bar (which silently shifts to mean TODAY once regular hours start -- see baseline_ out's history above for what trusting daily_bar here actually costs). Originally read via getattr(snap, "prev_daily_bar", None) -- alpaca-py's actual Snapshot field is `previous_daily_bar`, so that getattr's default silently fired every single time and this was empty from the day it shipped, with nothing to surface the typo since a missing value here fails soft by design (see below). Purely additive/optional context, never a required dependency -- per explicit instruction, its absence must never block or reject a symbol; fast_prediction_engine.py falls back to premarket high/ session high/detected support-resistance when this is missing. prev_close_out: [FEATURE 2026-09-16] same pattern as prev_day_high_out above -- {symbol: previous session's close} for breakout_scanner. py's gap_pct (needs the prior close as its denominator; unlike prev_day_high_out, breakout_scanner.py treats a missing value here as "can't score this symbol" rather than failing open, since gap% is a required, weighted input there -- see breakout_scanner.scan()). r premarket_scoringN price_min price_maxrmin_avg_daily_volumeg?previous_daily_bar)r get_snapshotsitems latest_trade daily_barrpricerr$rrrAttributeError TypeError)r5r<r=r>r?r6 scoring_cfgr: snapshotsr%snaprGrHrItoday_vol_so_farprev_bar prev_volumerrrprefilter_by_snapshot?sD 6      rRc Cstd}|dd}ttj}|t|ddd}||||}i}|D]0\}} | | d} t | dkr;q)| dd } | sDq)t | t d t | d d } | | d ||<q)|S)a [FEATURE 2026-09-15] {symbol: trailing-N-day average daily true range, as a % of price} -- one bulk multi-symbol daily-bars call (get_daily_bars_bulk, chunked) instead of one REST call per symbol. Feeds scorer.py's daily_atr_pct / universe.min_daily_atr_pct floor. Uses DAILY bars specifically, not the 1-min premarket/intraday bars already being pulled elsewhere -- real-world calibration (2026-09-15) found per-1-minute-bar ATR% does NOT separate a structurally dead BDC/SPAC from a real momentum name (both read ~0.1-0.3%, dominated by bar-granularity noise); only the multi-day range history does (WHF 2.23%, APXT 0.09% vs. 4.3%-21.9% for real same-day candidates). A symbol absent from the result (too new, delisted, or the request failed) is simply missing -- callers must treat that as "unknown," never as "confirmed low volatility" (see scorer.py's daily_atr_pct fail-open contract). r volatility_lookback_days )daysNr)periodgY@) rr-rnowrutcrget_daily_bars_bulkrFr0r min) r5r<r6 lookback_daysr\start daily_barsrr%rrIatr_valrrrcompute_volatility_baselines     rdcandidate_count prefilteredlookback_hoursvolume_baselinesprev_day_highsvolatility_baselinesc Cszt}|p |dd}t}td|dur5t|}tdt|t||}tdt|dtt j } | t |d} g} g} |D]} t | | | | }t|d krhtd | d t|d qHtd d|D}tdd|D}|| }|r|jrt|jnd}|r|jrt|jnd}|pi| ptdd}|pi| }t| |||||||d}||d<||d<|pi| |d<|dddstd | dqH|dddstd | dqH|dddstd | dqH|dddstd | d | |qH| |t| |qHtd!i}|d"dr| r|d#d$}t| |kr| jd%d&dd'|t| }| d|}|D]}d|dd(<| |t|d)|qXtd*t| t|d+t|d,t| | jd-d&dd'| d|}|D]}td |d)d.|d/qtd0t|d1t||S)2a| Full premarket scan. Returns a ranked list of scored candidate dicts, truncated to `candidate_count` (defaults to config's premarket_candidate_count, i.e. 20). [FEATURE 2026-08-17] `prefiltered` lets a caller (monitor.py) supply an already-computed universe+snapshot-filtered symbol list instead of this function pulling and filtering the full tradable-asset list again. monitor.py caches this list once at 09:00 and reuses it for every subsequent 30-minute intraday full rescan, so the expensive get_tradable_assets() + bulk snapshot call only ever happens once per day rather than once per rescan. `lookback_hours` defaults to 6 (covers the overnight/premarket session) but monitor.py's intraday rescans pass a much shorter window (intraday_health. full_rescan_lookback_hours, default 1) since a 6-hour window is unnecessary once the regular session is already underway. volume_baselines: [Phase 5 -- new, optional] {symbol: real_prior_day_ volume} from prefilter_by_snapshot()'s baseline_out -- when a symbol has a real entry here, its own historical volume is used for RVOL instead of the flat universe.min_avg_daily_volume constant. Omit (the default) to reproduce this function's exact pre-Phase-5 behavior. prev_day_highs: [FEATURE 2026-09-11] {symbol: previous session's high} from prefilter_by_snapshot()'s prev_day_high_out -- threaded through to each result dict as result["prev_day_high"] for fast_prediction_engine.py's resistance context. Optional, same fail-soft contract as volume_baselines above. volatility_baselines: [FEATURE 2026-09-15] {symbol: trailing-N-day avg daily ATR%} from compute_volatility_baseline() -- feeds scorer.py's daily_atr_pct floor (universe.min_daily_atr_pct), which hard-rejects structurally low-volatility names (BDCs, pre-merger SPACs) below. A symbol missing from this dict is "unknown," not "confirmed low volatility" -- never rejected on that basis alone, same fail-open contract as the other two baseline dicts above. r8premarket_candidate_countz[PREMARKET] Starting scanNz1[PREMARKET] Universe size after asset filtering: zREMARKET] Prefiltered to z symbols in price/volume band)hoursz [PREMARKET] z" rejected: insufficient bar data (z bars)cs|]}|dVqdS)rNrr&rrrr zscan..csro)rNrrprrrrqrrr rC) daily_atr_pctpm_highpm_low prev_day_highflagsmeets_min_volumeFz% rejected: below min premarket volumemeets_min_volatilityTzL rejected: below min daily ATR% (structurally low volatility, e.g. BDC/SPAC) spread_okz rejected: spread too widemeets_min_rvolz rejected: below min RVOLrvol_starvation_fallbackenabledmin_candidatescS|dSN total_scorerrrrrazscan..)keyreverservol_floor_waivedr%z+[PREMARKET] RVOL starvation fallback: only z+ candidate(s) cleared min_rvol; backfilled z1 more by score (rvol_floor_waived=True) to reach cSrrrrrrrror score=rz[PREMARKET] Selected r,)rrr3r4r;r0rRrr\rr]rrget_minute_barsdebugmaxr_get_latest_quote bid_pricer ask_pricer-rr data_storeappend_premarket_snapshotsortwarningwrite_premarket_candidates)rfrgrhrirjrkr6r5r r\rascoredrvol_near_missr%rrtruquotebidaskavg_vol_baselinersresult fallback_cfgr~needed backfilledrtoprrrscans*              " r__main__r%6srrz6.2fz price=metricsrIz.2fz rvol=rvol)NNN)NNreNNN)__doc__sysrrr config_loaderr logger_setupr alpaca_clientrscorerr indicatorsr r1rr3listrr;dictrRrdintrr__name__resultsrprintrrrrs\         i) *