一、元类
-
Python中，实例由类生成，类由type生成，所有类的最顶层基类是object，所有的类都继承自object，object类是type的实例，type是Python用来创建所有类的元类。元类是用来创建类的类。
-
普通类和元类创建类的区别：
- 普通类实例化时，先用
__new__构造新的空对象，然后调用__init__方法去初始化空对象。如果要调用__call__，需要使用实例后加一个括号显式调用。 - 元类创建类实例化时，首先调用元类的
__call__，然后才是__new__和__init__。由于要先调用__call__，可以控制类的生成，比如可以控制生成类的参数。
- 普通类实例化时，先用
-
backtrader中，基本上所有类从元类继承。MetaBase定义如下：
class MetaBase(type): def doprenew(cls, *args, **kwargs): return cls, args, kwargs def donew(cls, *args, **kwargs): _obj = cls.__new__(cls, *args, **kwargs)s return _obj, args, kwargs def dopreinit(cls, _obj, *args, **kwargs): return _obj, args, kwargs def doinit(cls, _obj, *args, **kwargs): _obj.__init__(*args, **kwargs) return _obj, args, kwargs def dopostinit(cls, _obj, *args, **kwargs): return _obj, args, kwargs def __call__(cls, *args, **kwargs): cls, args, kwargs = cls.doprenew(*args, **kwargs) _obj, args, kwargs = cls.donew(*args, **kwargs) _obj, args, kwargs = cls.dopreinit(_obj, *args, **kwargs) _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs) _obj, args, kwargs = cls.dopostinit(_obj, *args, **kwargs) return _obj-
MetaBase类定义了doprenew/donew/dopreinit/doinit/dopostinit方法，其中donew中调用类的
__new__创建对象，doinit函数调用类的__init__方法对对象进行初始化。而__call__函数直接完成预处理、实例化、初始化、后处理一系列动作，并且不需要显式调用，在类进行实例化时就会完成。 -
所有MetaBase(或者其子类)创建的类都会先调用
__call__函数，包括Cerebro、lineseries、strategies、broker、indicator等。
-
-
某些功能类并没有继承MetaBase，而是继承MetaBase的子类MetaParams，MetaParams定义如下：
class MetaParams(MetaBase): def __new__(meta, name, bases, dct): # Remove params from class definition to avoid inheritance # (and hence "repetition") newparams = dct.pop('params', ()) packs = 'packages' newpackages = tuple(dct.pop(packs, ())) # remove before creation fpacks = 'frompackages' fnewpackages = tuple(dct.pop(fpacks, ())) # remove before creation # Create the new class - this pulls predefined "params" cls = super(MetaParams, meta).__new__(meta, name, bases, dct) # Pulls the param class out of it - default is the empty class params = getattr(cls, 'params', AutoInfoClass) # Pulls the packages class out of it - default is the empty class packages = tuple(getattr(cls, packs, ())) fpackages = tuple(getattr(cls, fpacks, ())) # get extra (to the right) base classes which have a param attribute morebasesparams = [x.params for x in bases[1:] if hasattr(x, 'params')] # Get extra packages, add them to the packages and put all in the class for y in [x.packages for x in bases[1:] if hasattr(x, packs)]: packages += tuple(y) for y in [x.frompackages for x in bases[1:] if hasattr(x, fpacks)]: fpackages += tuple(y) cls.packages = packages + newpackages cls.frompackages = fpackages + fnewpackages # Subclass and store the newly derived params class cls.params = params._derive(name, newparams, morebasesparams) return cls def donew(cls, *args, **kwargs): clsmod = sys.modules[cls.__module__] # import specified packages for p in cls.packages: if isinstance(p, (tuple, list)): p, palias = p else: palias = p pmod = __import__(p) plevels = p.split('.') if p == palias and len(plevels) > 1: # 'os.path' not aliased setattr(clsmod, pmod.__name__, pmod) # set 'os' in module else: # aliased and/or dots for plevel in plevels[1:]: # recurse down the mod pmod = getattr(pmod, plevel) setattr(clsmod, palias, pmod) # import from specified packages - the 2nd part is a string or iterable for p, frompackage in cls.frompackages: if isinstance(frompackage, string_types): frompackage = (frompackage,) # make it a tuple for fp in frompackage: if isinstance(fp, (tuple, list)): fp, falias = fp else: fp, falias = fp, fp # assumed is string # complain "not string" without fp (unicode vs bytes) pmod = __import__(p, fromlist=[str(fp)]) pattr = getattr(pmod, fp) setattr(clsmod, falias, pattr) for basecls in cls.__bases__: setattr(sys.modules[basecls.__module__], falias, pattr) # Create params and set the values from the kwargs params = cls.params() for pname, pdef in cls.params._getitems(): setattr(params, pname, kwargs.pop(pname, pdef)) # Create the object and set the params in place _obj, args, kwargs = super(MetaParams, cls).donew(*args, **kwargs) _obj.params = params _obj.p = params # shorter alias # Parameter values have now been set before __init__ return _obj, args, kwargs- donew函数分别提取package、frompackage和params并赋值，params用于提取参数，对于backtrader很关键。
-
Cerebro类中，参数定义为元组，通过setattr直接赋值到具体独立参数，方便进行访问。
params = ( ('preload', True), ('runonce', True), ('maxcpus', None), ('stdstats', True), ('oldbuysell', False), ('oldtrades', False), ('lookahead', 0), ('exactbars', False), ('optdatas', True), ('optreturn', True), ('objcache', False), ('live', False), ('writer', False), ('tradehistory', False), ('oldsync', False), ('tz', None), ('cheat_on_open', False), ('broker_coo', True), ('quicknotify', False), ) -
元组在backtrader中的应用：
- 所有类实例化时通过元类的
__call__完成，不同的类可以进行一些特殊化的处理。 - 参数通过donew完成映射。
- 所有类实例化时通过元类的
-
元类在backtrader用于处理模板化工作，让具体功能类专注于功能实现。
二、Cerebro
1、Cerebro简介
-
Cerebro是backtrader的核心控制系统，构建了策略回测的基础框架，具体细节则在组件内部实现。Cerebro主要工作包括：
- 收集所有的输入(data feeds)，执行者(strategies)，观测者(Observers)、评价者(Analyzers)以及文档(Writers)，保证系统在任何时候都正常运行。
- 执行回测或者实时数据输入以及交易。
- 返回结果。
- 画图。
-
Cerebro类继承自MetaParams，代码如下：
class Cerebro(with_metaclass(MetaParams, object)): params = ( ('preload', True), ('runonce', True), ('maxcpus', None), ('stdstats', True), ('oldbuysell', False), ('oldtrades', False), ('lookahead', 0), ('exactbars', False), ('optdatas', True), ('optreturn', True), ('objcache', False), ('live', False), ('writer', False), ('tradehistory', False), ('oldsync', False), ('tz', None), ('cheat_on_open', False), ('broker_coo', True), ('quicknotify', False), ) def __init__(self): self._dolive = False self._doreplay = False self._dooptimize = False self.stores = list() self.feeds = list() self.datas = list() self.datasbyname = collections.OrderedDict() self.strats = list() self.optcbs = list() # holds a list of callbacks for opt strategies self.observers = list() self.analyzers = list() self.indicators = list() self.sizers = dict() self.writers = list() self.storecbs = list() self.datacbs = list() self.signals = list() self._signal_strat = (None, None, None) self._signal_concurrent = False self._signal_accumulate = False self._dataid = itertools.count(1) self._broker = BackBroker() self._broker.cerebro = self self._tradingcal = None # TradingCalendar() self._pretimers = list() self._ohistory = list() self._fhistory = None @staticmethod def iterize(iterable): def set_fund_history(self, fund): def add_order_history(self, orders, notify=True): def notify_timer(self, timer, when, *args, **kwargs): def add_timer(self, when, offset=datetime.timedelta(), repeat=datetime.timedelta(), weekdays=[], weekcarry=False, monthdays=[], monthcarry=True, allow=None, tzdata=None, strats=False, cheat=False, *args, **kwargs): def addtz(self, tz): def addcalendar(self, cal): def add_signal(self, sigtype, sigcls, *sigargs, **sigkwargs): def signal_strategy(self, stratcls, *args, **kwargs): def signal_concurrent(self, onoff): def signal_accumulate(self, onoff): def addstore(self, store): def addwriter(self, wrtcls, *args, **kwargs): def addsizer(self, sizercls, *args, **kwargs): def addsizer_byidx(self, idx, sizercls, *args, **kwargs): def addindicator(self, indcls, *args, **kwargs): def addanalyzer(self, ancls, *args, **kwargs): def addobserver(self, obscls, *args, **kwargs): def addobservermulti(self, obscls, *args, **kwargs): def addstorecb(self, callback): def notify_store(self, msg, *args, **kwargs): def adddatacb(self, callback): def notify_data(self, data, status, *args, **kwargs): def adddata(self, data, name=None): def chaindata(self, *args, **kwargs): def rolloverdata(self, *args, **kwargs): def replaydata(self, dataname, name=None, **kwargs): def resampledata(self, dataname, name=None, **kwargs): def optcallback(self, cb): def optstrategy(self, strategy, *args, **kwargs): def addstrategy(self, strategy, *args, **kwargs): def setbroker(self, broker): def getbroker(self): broker = property(getbroker, setbroker) def plot(self, plotter=None, numfigs=1, iplot=True, start=None, end=None, width=16, height=9, dpi=300, tight=True, use=None, **kwargs): def __call__(self, iterstrat): def __getstate__(self): def runstop(self): def run(self, **kwargs): def runstrategies(self, iterstrat, predata=False): def stop_writers(self, runstrats):
2、Cerebro初始化
-
Cerebro通过如下代码进行初始化：
cerebro = bt.Cerebro(**kwargs)-
首先调用Cerebro类的
__init__函数初始化类的属性。def __init__(self): self._dolive = False self._doreplay = False self._dooptimize = False self.stores = list() self.feeds = list() self.datas = list() self.datasbyname = collections.OrderedDict() self.strats = list() self.optcbs = list() # holds a list of callbacks for opt strategies self.observers = list() self.analyzers = list() self.indicators = list() self.sizers = dict() self.writers = list() self.storecbs = list() self.datacbs = list() self.signals = list() self._signal_strat = (None, None, None) self._signal_concurrent = False self._signal_accumulate = False self._dataid = itertools.count(1) self._broker = BackBroker()#初始化一个缺省的broker实例。 self._broker.cerebro = self self._tradingcal = None # TradingCalendar() self._pretimers = list() self._ohistory = list() self._fhistory = None名称 定义 说明 stores list 用于存储接收到的其它组件(例如data)消息。 feeds list 用于存储数据源(data Feeds)的基类。 datas list 用于存储数据源。feeds是data的基类。 datasbyname OrderedDict 存储数据源的名称，存储的方式是有序的字典。 strats list 用于存储策略(strategies)类 optcbs list 存储优化策略的回调。对执行完的优化策略进行处理的回调。 observers list 保存observers类 analyzers list 保存analyzers类 indicators list 保存indicators类 writers list 保存writers类 storecbs list 保存对notify_store消息处理的回调.对于收到存储在store消息中可以定制处理过程。 datacbs list 保存对data类消息处理的回调 signals list 保存信号，用于通过signal strategy进行回测的机制。 sizers dict 保存sizers类 -
Cerebro类还支持一系列的参数(在元类中统一进行处理)，实例化的时候通过
**kwargs传递，参数以元组的元组形式定义，如下：params = ( ('preload', True), ('runonce', True), ('maxcpus', None), ('stdstats', True), ('oldbuysell', False), ('oldtrades', False), ('lookahead', 0), ('exactbars', False), ('optdatas', True), ('optreturn', True), ('objcache', False), ('live', False), ('writer', False), ('tradehistory', False), ('oldsync', False), ('tz', None), ('cheat_on_open', False), ('broker_coo', True), ('quicknotify', False), )参数 取值范围 缺省值 含义 preload True False TRUE 是否为Strategies预加载传递给Cerebro的不同数据源。 runonce True False TRUE runonce是indicators类对数据访问方法的优化以加快速度，使用矢量化模式来提高算法的运行速度。strategies和Observers通常是基于事件对数据进行处理。 maxcpus None->运行系统可用的核 None 指定可以用于优化的CPU核 stdstats True False True 如果为True，创建缺省Observers，包括Broker、Buysell和Trades。 oldbuysell True False FALSE 创建Observers时，使用新buysell还是旧版本实现，没有特殊需求，使用新代码。 oldtrades True False FALSE 创建Observers时，使用新buysell还是旧版本实现，没有特殊需求，使用新代码。 lookahead 数值 0 用于扩展数据时，扩大数据的缓存的大小。通常设置缺省值即可。 exactbars False，数值 FALSE 用于指示如何缓存数据以节约内存。False是Lines对象数据都会加入到内存以方便计算。如果采用其它值，会有一些特殊处理减少内存，可能对画图、runonce等机制造成影响。缺省False。 optdatas True False True 速度处理的优化机制，主要是数据预装载、runonce等，可以提高效率。 optreturn True False True 对不同类的优化机制，可以提高效率，设置为缺省值即可。 objcache True False False 实验性质的方法，没啥用，缺省值 live True False False 是否实时输入数据。 writer True False False 设置为True，为自动创建一个writer，缺省输出日志到stdout。 tradehistory True False False 设置为True，会记录所有策略的每一次交易信息。也可以在strategies里面设置 oldsync True False False 新版本后(1.9.0.99之后)提供新来的datas同步机制。如果要用老的同步机制，可以设置为True。 tz None，String None 记录时区信息，缺省None，使用的就是UTC，对于中国是’UTC+8’。对于回测，时区并不重要。 cheat_on_open True False False 设置为True，会调用strategies的next_open方法，在next前调用，主要用于在对订单的评估，可以基于前一天的open价发起一个订单。 broker_coo True False True 设置为True，broker调用set_coo开启’cheat_on_open’，此时cheat_on_open也必须打开。 quicknotify True False False 在next前发起broker的通知。只有实时数据可以快速通知
-
3、Cerebro添加数据源
-
增加数据源最常见的方式就是cerebro.adddata(data)，data必须是已经实例化的data feed。
data = bt.feeds.PandasData(dataname=stock_hfq_df, fromdate=start_date, todate=end_date) # 加载数据 cerebro.adddata(data) -
也可以通过resample和replay添加数据源：
cerebro.resampledata(data,timeframe=bt.TimeFrame.Weeks) cerebro.replaydatadata(data, timeframe=bt.TimeFrame.Days) -
backtrader可以接受任何数量的data feeds，包括普通数据以及resample/replay的数据。
-
adddata代码如下：
def adddata(self, data, name=None): ''' Adds a ``Data Feed`` instance to the mix. If ``name`` is not None it will be put into ``data._name`` which is meant for decoration/plotting purposes. ''' if name is not None: data._name = name data._id = next(self._dataid) data.setenvironment(self) self.datas.append(data) self.datasbyname[data._name] = data feed = data.getfeed() if feed and feed not in self.feeds: self.feeds.append(feed) if data.islive(): self._dolive = True return data- 记录下data的名称
- 分配data的ID，从1开始。
- Cerebro实例和data建立关联关系：data调用setenvironment记录自己现在归当前Cerebro管，同时Cerebro将data加入到datas列表中。
- 获取data的feed，如果feed有效，加入到feeds列表中。feed是data的基类。
- 记录是否实时数据。
-
resampdata的代码如下：
def resampledata(self, dataname, name=None, **kwargs): ''' Adds a ``Data Feed`` to be resample by the system If ``name`` is not None it will be put into ``data._name`` which is meant for decoration/plotting purposes. Any other kwargs like ``timeframe``, ``compression``, ``todate`` which are supported by the resample filter will be passed transparently ''' if any(dataname is x for x in self.datas): dataname = dataname.clone() dataname.resample(**kwargs) self.adddata(dataname, name=name) self._doreplay = True return dataname- 判断resample目的dataname(data对象实例)是否已经保存在datas列表中。如果已经保存，那么不能直接resample，需要clone一个新的data对象。
- 调用data feeds的resample函数对数据进行处理。
- 直接调用adddata函数处理，同时记录._doreplay标记。
-
replaydata的代码：
def replaydata(self, dataname, name=None, **kwargs): ''' Adds a ``Data Feed`` to be replayed by the system If ``name`` is not None it will be put into ``data._name`` which is meant for decoration/plotting purposes. Any other kwargs like ``timeframe``, ``compression``, ``todate`` which are supported by the replay filter will be passed transparently ''' if any(dataname is x for x in self.datas): dataname = dataname.clone() dataname.replay(**kwargs) self.adddata(dataname, name=name) self._doreplay = True return dataname
4、Cerebro添加策略
-
Cerebro添加策略代码如下：
cerebro.addstrategy(TestStrategy) cerebro.optstrategy(TestStrategy, maperiod=range(3,15)) -
addstrategy代码实现：
def addstrategy(self, strategy, *args, **kwargs): ''' Adds a ``Strategy`` class to the mix for a single pass run. Instantiation will happen during ``run`` time. args and kwargs will be passed to the strategy as they are during instantiation. Returns the index with which addition of other objects (like sizers) can be referenced ''' self.strats.append([(strategy, args, kwargs)]) return len(self.strats) - 1- 将strategy类及其参数加入到Cerebro中用于存储策略的容器中(strats)，返回索引。strategies只有在run时才会实例化。
-
optstrategy实现代码如下：
def optstrategy(self, strategy, *args, **kwargs): ''' Adds a ``Strategy`` class to the mix for optimization. Instantiation will happen during ``run`` time. args and kwargs MUST BE iterables which hold the values to check. Example: if a Strategy accepts a parameter ``period``, for optimization purposes the call to ``optstrategy`` looks like: - cerebro.optstrategy(MyStrategy, period=(15, 25)) This will execute an optimization for values 15 and 25. Whereas - cerebro.optstrategy(MyStrategy, period=range(15, 25)) will execute MyStrategy with ``period`` values 15 -> 25 (25 not included, because ranges are semi-open in Python) If a parameter is passed but shall not be optimized the call looks like: - cerebro.optstrategy(MyStrategy, period=(15,)) Notice that ``period`` is still passed as an iterable ... of just 1 element ``backtrader`` will anyhow try to identify situations like: - cerebro.optstrategy(MyStrategy, period=15) and will create an internal pseudo-iterable if possible ''' self._dooptimize = True args = self.iterize(args) optargs = itertools.product(*args) optkeys = list(kwargs) vals = self.iterize(kwargs.values()) optvals = itertools.product(*vals) okwargs1 = map(zip, itertools.repeat(optkeys), optvals) optkwargs = map(dict, okwargs1) it = itertools.product([strategy], optargs, optkwargs) self.strats.append(it)
5、Cerebro更改Broker
-
Cerebro初始化时实例化了一个缺省broker，如果需要提供自己的broker，可以通过如下方法更改：
broker = MyBroker() cerebro.broker = broker -
Cerebro代码实现中提供了property装饰器
broker = property(getbroker, setbroker)，等同通过setbroker/getbroker设置/读取。
6、Cerebro添加评价指标
-
Cerebro添加评价指标代码如下：
tframes = dict( days=bt.TimeFrame.Days, weeks=bt.TimeFrame.Weeks, months=bt.TimeFrame.Months, years=bt.TimeFrame.Years) cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='SharpeRatio') cerebro.addanalyzer(bt.analyzers.DrawDown,_name = 'DrawDown') cerebro.addanalyzer(bt.analyzers.AnnualReturn,_name = 'AnnualReturn') cerebro.addanalyzer(bt.analyzers.SQN,_name = 'SQN') cerebro.addanalyzer(bt.analyzers.TradeAnalyzer,_name = 'TradeAnalyzer') cerebro.addanalyzer(bt.analyzers.PositionsValue,_name = 'PositionsValue') cerebro.addanalyzer(bt.analyzers.Returns,_name = 'Returns') cerebro.addanalyzer(bt.analyzers.LogReturnsRolling,timeframe=tframes['years'],_name = 'LogReturnsRolling') cerebro.addanalyzer(bt.analyzers.Transactions, _name='Transactions') -
addanalyzer实现代码如下：
def addanalyzer(self, ancls, *args, **kwargs): ''' Adds an ``Analyzer`` class to the mix. Instantiation will be done at ``run`` time ''' self.analyzers.append((ancls, args, kwargs))
7、Cerebro添加观测器
-
Cerebro添加观测器代码如下：
cerebro.addobserver(bt.observers.Broker) cerebro.addobserver(bt.observers.Trades) cerebro.addobserver(bt.observers.BuySell) -
addobserver实现代码：
def addobserver(self, obscls, *args, **kwargs): ''' Adds an ``Observer`` class to the mix. Instantiation will be done at ``run`` time ''' self.observers.append((False, obscls, args, kwargs)) def addobservermulti(self, obscls, *args, **kwargs): ''' Adds an ``Observer`` class to the mix. Instantiation will be done at ``run`` time It will be added once per "data" in the system. A use case is a buy/sell observer which observes individual datas. A counter-example is the CashValue, which observes system-wide values ''' self.observers.append((True, obscls, args, kwargs))
8、Cerebro运行
-
运行代码如下：
results = cerebro.run(**kwargs) -
Cerebro运行参数如下：
-
preload：是否为Strategies预加载传递给Cerebro的不同数据源，默认为False。
-
runonce：是否以矢量化模式运行Indicators以加速整个系统，默认为True。
-
live：是否为实时数据，默认为False，是回测数据。实时数据时会停用preload和runonce，但对内存节省方案没有影响。
-
maxcpus：同时使用多少个内核进行优化，默认值为None，表示所有可用CPU。
-
stdstats：是否添加默认观测器：Broker、Trade和BuySell，默认值为True。
-
objcache：是否为line对象的缓存，默认为False。
-
writer：是否将标准信息的输出生成一个默认文件，默认为False。
-
tradehistory：是否将所有策略的每笔交易中激活更新事件记录log，默认False。
-
optdatas：是否进行数据优化，数据预加载将在主进程中只进行一次，以节省时间和资源，默认为True，但需要preload和runonce也为True。
-
optreturn：是否只优化params属性和analyzers，可以在一定程度优化速度，默认为True。
-
oldsync：是否使用旧版本数据同步，默认为False。从版本1.9.0.99开始，多个数据(相同或不同时间范围)的同步已更改为允许不同长度的数据。
-
tz：为策略添加全球时区，默认为None。
-
None：策略显示日期时间将采用UTC。
-
pytz 实例：将UTC时间转换为所选时区
-
string：将尝试实例化pytz实例。
-
-
cheat_on_open：是否使用cheat_on_open订单，默认为False。当为True时next_open调用发生在next方法调用前，此时指标尚未重新计算，允许发送一个考虑前一天指标但使用open价格计算的订单。
-
broker_coo：是否自动调用set_coo代理的方法来激活cheat_on_open执行，默认为True。
-
quicknotify：Broker通知在下一个价格交付前交付 。对于回溯测试，没有任何影响；对于实盘Broker，可以在bar交付前很久就发出通知。
-
-
run代码实现如下：
def run(self, **kwargs): '''The core method to perform backtesting. Any ``kwargs`` passed to it will affect the value of the standard parameters ``Cerebro`` was instantiated with. If ``cerebro`` has not datas the method will immediately bail out. It has different return values: - For No Optimization: a list contanining instances of the Strategy classes added with ``addstrategy`` - For Optimization: a list of lists which contain instances of the Strategy classes added with ``addstrategy`` ''' self._event_stop = False # Stop is requested if not self.datas: return [] # nothing can be run pkeys = self.params._getkeys() for key, val in kwargs.items(): if key in pkeys: setattr(self.params, key, val) # Manage activate/deactivate object cache linebuffer.LineActions.cleancache() # clean cache indicator.Indicator.cleancache() # clean cache linebuffer.LineActions.usecache(self.p.objcache) indicator.Indicator.usecache(self.p.objcache) self._dorunonce = self.p.runonce self._dopreload = self.p.preload self._exactbars = int(self.p.exactbars) if self._exactbars: self._dorunonce = False # something is saving memory, no runonce self._dopreload = self._dopreload and self._exactbars < 1 self._doreplay = self._doreplay or any(x.replaying for x in self.datas) if self._doreplay: # preloading is not supported with replay. full timeframe bars # are constructed in realtime self._dopreload = False if self._dolive or self.p.live: # in this case both preload and runonce must be off self._dorunonce = False self._dopreload = False self.runwriters = list() # Add the system default writer if requested if self.p.writer is True: wr = WriterFile() self.runwriters.append(wr) # Instantiate any other writers for wrcls, wrargs, wrkwargs in self.writers: wr = wrcls(*wrargs, **wrkwargs) self.runwriters.append(wr) # Write down if any writer wants the full csv output self.writers_csv = any(map(lambda x: x.p.csv, self.runwriters)) self.runstrats = list() if self.signals: # allow processing of signals signalst, sargs, skwargs = self._signal_strat if signalst is None: # Try to see if the 1st regular strategy is a signal strategy try: signalst, sargs, skwargs = self.strats.pop(0) except IndexError: pass # Nothing there else: if not isinstance(signalst, SignalStrategy): # no signal ... reinsert at the beginning self.strats.insert(0, (signalst, sargs, skwargs)) signalst = None # flag as not presetn if signalst is None: # recheck # Still None, create a default one signalst, sargs, skwargs = SignalStrategy, tuple(), dict() # Add the signal strategy self.addstrategy(signalst, _accumulate=self._signal_accumulate, _concurrent=self._signal_concurrent, signals=self.signals, *sargs, **skwargs) if not self.strats: # Datas are present, add a strategy self.addstrategy(Strategy) iterstrats = itertools.product(*self.strats) if not self._dooptimize or self.p.maxcpus == 1: # If no optimmization is wished ... or 1 core is to be used # let's skip process "spawning" for iterstrat in iterstrats: runstrat = self.runstrategies(iterstrat) self.runstrats.append(runstrat) if self._dooptimize: for cb in self.optcbs: cb(runstrat) # callback receives finished strategy else: if self.p.optdatas and self._dopreload and self._dorunonce: for data in self.datas: data.reset() if self._exactbars < 1: # datas can be full length data.extend(size=self.params.lookahead) data._start() if self._dopreload: data.preload() pool = multiprocessing.Pool(self.p.maxcpus or None) for r in pool.imap(self, iterstrats): self.runstrats.append(r) for cb in self.optcbs: cb(r) # callback receives finished strategy pool.close() if self.p.optdatas and self._dopreload and self._dorunonce: for data in self.datas: data.stop() if not self._dooptimize: # avoid a list of list for regular cases return self.runstrats[0] return self.runstrats- 记录_event_stop为False，标识开始启动，需要通过stop来停止。
- 检查是否有数据源(datas)，没有则退出。
- 如果run函数携带参数是否在Cerebro的参数表中，更新Cerebro实例的参数。Cerebro参数在初始化时可以设置，在run时也可以设置。
- 处理参数。根据参数设置一些标记，清空缓存，为系统运行做好准备。
- 如果参数标识要求提供writers(记录日志)，有则实例化一个缺省writers。同时检查是否还有通过addwriter添加的其它的writers，有则实例化。将writers加入到runwriters容器，然后检查writers有没有参数要求输出csv，任意writers要求输出CSV，则记录到writers_csv标识。
- 初始化runstrats策略运行容器。
- 查看_signal_strat容器中有没有记录信号策略类(SignalStrategy，通过signal_strategy加入)。如果没有，则查看普通strategies容器strats第一个策略是不是SignalStrategy类；如果不是，返回SignalStrategy，创建一个空的signal_strategy。用户自己通过signal_strategy函数添加的SignalStrategy，普通strategy容器中的SignalStrategy以及缺省创建的SignalStrategy，三个按优先顺序，必须插入一个信号策略类，保存在普通strategies容器strats中，只是参数设置为signal。如果要使用信号回测方式，那么一定要加一个信号策略类。
- 如果用户没有设定策略，Cerebro添加一个缺省strategies类。
- 使用product将所有的strategies类存放到iterstrats迭代器中。
- 如果不进行性能优化或者参数中maxcpus为1，那么直接按循序调用runstrategies执行策略。如果是优化策略(通过optstrategy添加的策略)，策略执行完成还可以调用回调(回调可通过optcallback增加)进行处理。
- 如果需要进行性能优化处理：首先对数据进行reset/start/preload处理，然后根据maxcpus参数启用多进程，在多进程中调用runstrategies进行策略运行。对于优化策略，策略执行完毕时进行回调处理。策略执行完毕时调用data的stop方法。
9、策略运行机制
-
runstrategies函数代码如下：
def runstrategies(self, iterstrat, predata=False): ''' Internal method invoked by ``run```to run a set of strategies ''' self._init_stcount() self.runningstrats = runstrats = list() for store in self.stores: store.start() if self.p.cheat_on_open and self.p.broker_coo: # try to activate in broker if hasattr(self._broker, 'set_coo'): self._broker.set_coo(True) if self._fhistory is not None: self._broker.set_fund_history(self._fhistory) for orders, onotify in self._ohistory: self._broker.add_order_history(orders, onotify) self._broker.start() for feed in self.feeds: feed.start() if self.writers_csv: wheaders = list() for data in self.datas: if data.csv: wheaders.extend(data.getwriterheaders()) for writer in self.runwriters: if writer.p.csv: writer.addheaders(wheaders) # self._plotfillers = [list() for d in self.datas] # self._plotfillers2 = [list() for d in self.datas] if not predata: for data in self.datas: data.reset() if self._exactbars < 1: # datas can be full length data.extend(size=self.params.lookahead) data._start() if self._dopreload: data.preload() for stratcls, sargs, skwargs in iterstrat: sargs = self.datas + list(sargs) try: strat = stratcls(*sargs, **skwargs) except bt.errors.StrategySkipError: continue # do not add strategy to the mix if self.p.oldsync: strat._oldsync = True # tell strategy to use old clock update if self.p.tradehistory: strat.set_tradehistory() runstrats.append(strat) tz = self.p.tz if isinstance(tz, integer_types): tz = self.datas[tz]._tz else: tz = tzparse(tz) if runstrats: # loop separated for clarity defaultsizer = self.sizers.get(None, (None, None, None)) for idx, strat in enumerate(runstrats): if self.p.stdstats: strat._addobserver(False, observers.Broker) if self.p.oldbuysell: strat._addobserver(True, observers.BuySell) else: strat._addobserver(True, observers.BuySell, barplot=True) if self.p.oldtrades or len(self.datas) == 1: strat._addobserver(False, observers.Trades) else: strat._addobserver(False, observers.DataTrades) for multi, obscls, obsargs, obskwargs in self.observers: strat._addobserver(multi, obscls, *obsargs, **obskwargs) for indcls, indargs, indkwargs in self.indicators: strat._addindicator(indcls, *indargs, **indkwargs) for ancls, anargs, ankwargs in self.analyzers: strat._addanalyzer(ancls, *anargs, **ankwargs) sizer, sargs, skwargs = self.sizers.get(idx, defaultsizer) if sizer is not None: strat._addsizer(sizer, *sargs, **skwargs) strat._settz(tz) strat._start() for writer in self.runwriters: if writer.p.csv: writer.addheaders(strat.getwriterheaders()) if not predata: for strat in runstrats: strat.qbuffer(self._exactbars, replaying=self._doreplay) for writer in self.runwriters: writer.start() # Prepare timers self._timers = [] self._timerscheat = [] for timer in self._pretimers: # preprocess tzdata if needed timer.start(self.datas[0]) if timer.params.cheat: self._timerscheat.append(timer) else: self._timers.append(timer) if self._dopreload and self._dorunonce: if self.p.oldsync: self._runonce_old(runstrats) else: self._runonce(runstrats) else: if self.p.oldsync: self._runnext_old(runstrats) else: self._runnext(runstrats) for strat in runstrats: strat._stop() self._broker.stop() if not predata: for data in self.datas: data.stop() for feed in self.feeds: feed.stop() for store in self.stores: store.stop() self.stop_writers(runstrats) if self._dooptimize and self.p.optreturn: # Results can be optimized results = list() for strat in runstrats: for a in strat.analyzers: a.strategy = None a._parent = None for attrname in dir(a): if attrname.startswith('data'): setattr(a, attrname, None) oreturn = OptReturn(strat.params, analyzers=strat.analyzers, strategycls=type(strat)) results.append(oreturn) return results return runstrats-
策略运行计数。
-
初始化runningstrats容器，用于存储正在运行的策略。
-
对组件参数设置和初始化：启动信号存储(store)；broker参数设置和启动；启动feed；设置Writers参数。
-
如果predata为False，调用preload预加载数据。部分性能优化情况下，在run时就执行preload。
-
从iterstrat迭代器(run调用runningstrats函数携带参数)提取每一个strategy和参数，进行strategy的实例化和初始化，并返回正在执行的strategy的实例，记录到runstrats容器中。策略对象实例化使用元类执行：
strat = stratcls(*sargs, **skwargs) -
记录下时区，用datas类存储的时区或者参数输入时区。
-
针对strategy实例分别加入Observers、indicators、analyzer以及sizer类，并调用start函数开始启动。
-
针对writers(存储在runwriters中，run时实例化)，设置表头，并启动。
-
设定定时器，并对strategies执行
_runonce/_runonce_old/_runnext_old/_runnexth函数。 -
停止各个组件。对于优化策略，需要对运行结果进行分析。
-
-
_runnext函数如下：def _runnext(self, runstrats): ''' Actual implementation of run in full next mode. All objects have its ``next`` method invoke on each data arrival ''' datas = sorted(self.datas, key=lambda x: (x._timeframe, x._compression)) datas1 = datas[1:] data0 = datas[0] d0ret = True rs = [i for i, x in enumerate(datas) if x.resampling] rp = [i for i, x in enumerate(datas) if x.replaying] rsonly = [i for i, x in enumerate(datas) if x.resampling and not x.replaying] onlyresample = len(datas) == len(rsonly) noresample = not rsonly clonecount = sum(d._clone for d in datas) ldatas = len(datas) ldatas_noclones = ldatas - clonecount lastqcheck = False dt0 = date2num(datetime.datetime.max) - 2 # default at max while d0ret or d0ret is None: # if any has live data in the buffer, no data will wait anything newqcheck = not any(d.haslivedata() for d in datas) if not newqcheck: # If no data has reached the live status or all, wait for # the next incoming data livecount = sum(d._laststatus == d.LIVE for d in datas) newqcheck = not livecount or livecount == ldatas_noclones lastret = False # Notify anything from the store even before moving datas # because datas may not move due to an error reported by the store self._storenotify() if self._event_stop: # stop if requested return self._datanotify() if self._event_stop: # stop if requested return # record starting time and tell feeds to discount the elapsed time # from the qcheck value drets = [] qstart = datetime.datetime.utcnow() for d in datas: qlapse = datetime.datetime.utcnow() - qstart d.do_qcheck(newqcheck, qlapse.total_seconds()) drets.append(d.next(ticks=False)) d0ret = any((dret for dret in drets)) if not d0ret and any((dret is None for dret in drets)): d0ret = None if d0ret: dts = [] for i, ret in enumerate(drets): dts.append(datas[i].datetime[0] if ret else None) # Get index to minimum datetime if onlyresample or noresample: dt0 = min((d for d in dts if d is not None)) else: dt0 = min((d for i, d in enumerate(dts) if d is not None and i not in rsonly)) dmaster = datas[dts.index(dt0)] # and timemaster self._dtmaster = dmaster.num2date(dt0) self._udtmaster = num2date(dt0) # slen = len(runstrats[0]) # Try to get something for those that didn't return for i, ret in enumerate(drets): if ret: # dts already contains a valid datetime for this i continue # try to get a data by checking with a master d = datas[i] d._check(forcedata=dmaster) # check to force output if d.next(datamaster=dmaster, ticks=False): # retry dts[i] = d.datetime[0] # good -> store # self._plotfillers2[i].append(slen) # mark as fill else: # self._plotfillers[i].append(slen) # mark as empty pass # make sure only those at dmaster level end up delivering for i, dti in enumerate(dts): if dti is not None: di = datas[i] rpi = False and di.replaying # to check behavior if dti > dt0: if not rpi: # must see all ticks ... di.rewind() # cannot deliver yet # self._plotfillers[i].append(slen) elif not di.replaying: # Replay forces tick fill, else force here di._tick_fill(force=True) # self._plotfillers2[i].append(slen) # mark as fill elif d0ret is None: # meant for things like live feeds which may not produce a bar # at the moment but need the loop to run for notifications and # getting resample and others to produce timely bars for data in datas: data._check() else: lastret = data0._last() for data in datas1: lastret += data._last(datamaster=data0) if not lastret: # Only go extra round if something was changed by "lasts" break # Datas may have generated a new notification after next self._datanotify() if self._event_stop: # stop if requested return if d0ret or lastret: # if any bar, check timers before broker self._check_timers(runstrats, dt0, cheat=True) if self.p.cheat_on_open: for strat in runstrats: strat._next_open() if self._event_stop: # stop if requested return self._brokernotify() if self._event_stop: # stop if requested return if d0ret or lastret: # bars produced by data or filters self._check_timers(runstrats, dt0, cheat=False) for strat in runstrats: strat._next() if self._event_stop: # stop if requested return self._next_writers(runstrats) # Last notification chance before stopping self._datanotify() if self._event_stop: # stop if requested return self._storenotify() if self._event_stop: # stop if requested return- 按照周期大小排序datas，将具有最小周期的data作为data0，其它放到datas1容器中。
- 将resample和replay的数据索引单独保存，做好标记。计算clone数据(加入resample和replay数据时都会clone)的个数、非clone数据的个数以及数据总的数量。
- dt0初始化最大值为最大日期(9999年12月31日)的多边格里高利度序数(每一日期对应一个独一无二的数，方便程序处理)。
- 如果所有数据是live或者没有新数据，需要等待。
- 识别有没有收到stop消息（从store容器中读取），在等待数据的时候，可能因为失败没有进一步的数据。
- 所有的data都会调用do_qcheck以及next。
- 后面一段确实比较难以理解，其总体思路是寻找主data（dmaster），通常应该是周期最短的数据。然后其他数据以此数据作为基准。（目前理解有限，后续在解读Data相关类的时候针对此场景再补充分析）
在此过程中，需要调用_brokernotify和_datanotify检测是否有停止信号。 - 数据完备（bar成功产生之后），就调用策略的
_next方法，这样就进入到我们定制策略next中进行处理了。
-
_runonce函数实现：
def _runonce(self, runstrats): ''' Actual implementation of run in vector mode. Strategies are still invoked on a pseudo-event mode in which ``next`` is called for each data arrival ''' for strat in runstrats: strat._once() strat.reset() # strat called next by next - reset lines # The default once for strategies does nothing and therefore # has not moved forward all datas/indicators/observers that # were homed before calling once, Hence no "need" to do it # here again, because pointers are at 0 datas = sorted(self.datas, key=lambda x: (x._timeframe, x._compression)) while True: # Check next incoming date in the datas dts = [d.advance_peek() for d in datas] dt0 = min(dts) if dt0 == float('inf'): break # no data delivers anything # Timemaster if needed be # dmaster = datas[dts.index(dt0)] # and timemaster slen = len(runstrats[0]) for i, dti in enumerate(dts): if dti <= dt0: datas[i].advance() # self._plotfillers2[i].append(slen) # mark as fill else: # self._plotfillers[i].append(slen) pass self._check_timers(runstrats, dt0, cheat=True) if self.p.cheat_on_open: for strat in runstrats: strat._oncepost_open() if self._event_stop: # stop if requested return self._brokernotify() if self._event_stop: # stop if requested return self._check_timers(runstrats, dt0, cheat=False) for strat in runstrats: strat._oncepost(dt0) if self._event_stop: # stop if requested return self._next_writers(runstrats)- 针对每个strategy，运行
_once方法并执行reset，将数据索引指向0。 - 将数据按照周期(timeframe)大小排序。
- 调用advance_peek看下一个时间(索引为1)，最近时间记为dt0。
- 对最近时间的data调用advance处理。
- 如果设置cheat_on_open标识，在策略的next前调用next_open方法。
- 检测是否收到broker的stop消息。
- 调用策略
_oncepost方法，会调用到strategy的next方法。
- 针对每个strategy，运行
10、可视化
-
调用plot方法进行可视化输出，plot方法实现：
def plot(self, plotter=None, numfigs=1, iplot=True, start=None, end=None, width=16, height=9, dpi=300, tight=True, use=None, **kwargs): ''' Plots the strategies inside cerebro If ``plotter`` is None a default ``Plot`` instance is created and ``kwargs`` are passed to it during instantiation. ``numfigs`` split the plot in the indicated number of charts reducing chart density if wished ``iplot``: if ``True`` and running in a ``notebook`` the charts will be displayed inline ``use``: set it to the name of the desired matplotlib backend. It will take precedence over ``iplot`` ``start``: An index to the datetime line array of the strategy or a ``datetime.date``, ``datetime.datetime`` instance indicating the start of the plot ``end``: An index to the datetime line array of the strategy or a ``datetime.date``, ``datetime.datetime`` instance indicating the end of the plot ``width``: in inches of the saved figure ``height``: in inches of the saved figure ``dpi``: quality in dots per inches of the saved figure ``tight``: only save actual content and not the frame of the figure ''' if self._exactbars > 0: return if not plotter: from . import plot if self.p.oldsync: plotter = plot.Plot_OldSync(**kwargs) else: plotter = plot.Plot(**kwargs) # pfillers = {self.datas[i]: self._plotfillers[i] # for i, x in enumerate(self._plotfillers)} # pfillers2 = {self.datas[i]: self._plotfillers2[i] # for i, x in enumerate(self._plotfillers2)} figs = [] for stratlist in self.runstrats: for si, strat in enumerate(stratlist): rfig = plotter.plot(strat, figid=si * 100, numfigs=numfigs, iplot=iplot, start=start, end=end, use=use) # pfillers=pfillers2) figs.append(rfig) plotter.show() return figs -
引入plotter模块，针对每一个运行的strategy进行绘图。
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
