A translational platform for prototyping closed-loop neuromodulation systems.

3.3 MB·English

by  Afshar, Pedram

METHODSARTICLE
NEURAL CIRCUITS published:22 January2013
doi:10.3389/fncir.2012.00117
A translational platform for prototyping closed-loop
neuromodulation systems
PedramAfshar*,AnkitKhambhati,ScottStanslaski,David Carlson,RandyJensen,DaveLinde,
SiddharthDani,MaciejLazarewicz,PengCong,JonGiftakis,PaulStypulkowski andTimDenison*
MedtronicNeuromodulation,Minneapolis,MN,USA
Editedby: While modulating neural activity through stimulation is an effective treatment for
SteveM.Potter,GeorgiaInstituteof neurological diseases such as Parkinson’s disease and essential tremor, an opportunity
Technology,USA for improving neuromodulation therapy remains in automatically adjusting therapy to
Reviewedby: continuously optimize patient outcomes. Practical issues associated with achieving this
ThomasDeMarse,Universityof
include the paucity of human data related to disease states, poorly validated estimators
Florida,USA
DouglasJ.Bakkum,Georgia of patient state, and unknown dynamic mappings of optimal stimulation parameters
InstituteofTechnology,USA basedonestimatedstates.Toovercomethesechallenges,wepresentaninvestigational
*Correspondence: platform including: an implanted sensing and stimulation device to collect data and
PedramAfsharandTimDenison, run automated closed-loop algorithms; an external tool to prototype classifier and
MedtronicNeuromodulation,
control-policyalgorithms;andreal-timetelemetrytoupdatetheimplanteddevicefirmware
7000CentralAvenueNorth,
Minneapolis,MN55432,USA. andmonitoritsstate.Theprototypingsystemwasdemonstratedinachroniclargeanimal
e-mail:timothy.denison@ model studying hippocampal dynamics. We used the platform to find biomarkers of the
medtronic.com; observed states and transfer functions of different stimulation amplitudes. Data showed
[email protected]
that moderatelevels ofstimulationsuppresshippocampalbetaactivity,whilehighlevels
of stimulation produce seizure-like after-discharge activity. The biomarker and transfer
function observations were mapped into classifier and control-policy algorithms, which
were downloaded to the implanted device to continuously titrate stimulation amplitude
for the desired network effect. The platform is designed to be a flexible prototyping tool
and could be used to develop improved mechanistic models and automated closed-loop
systemsforavarietyofneurologicaldisorders.
Keywords:automation,closed-loop,neuromodulation,prototyping,hippocampus,seizure
INTRODUCTION intoestimatesofstate(i.e.,aclassifieralgorithm)andanotherthat
Neuromodulation devices for deep brain stimulation (DBS) translatesthestateestimateintoastimulationparameterupdate
deliver targeted electrical stimulation to treat symptoms of dis- (i.e.,acontrol-policyalgorithm).Inthiswork,stateisleftinten-
easessuchasParkinson’sdisease,essentialtremor,anddystonia. tionally ambiguous because its meaning depends on the appli-
Toensurebenefit,thesetherapiesrequirenotonlyaccurateplace- cation:examplesincludeseizureversusnon-seizure;Parkinson’s
ment of the stimulating electrode within neural tissue, but also ON versus OFF; asleep versus awake; or others. Regardless of
properselectionofstimulationparameters(e.g.,amplitude,pulse the application, dividing the algorithm provides the following
width,andfrequency).Theseparameterscanbeusedtomitigate benefits:
sideeffectsincludinghemiballism,gaitandspeechdisturbances,
and dyskinesias (Limousin et al., 1996, 1998; Hamani et al., • Matches clinical workflow: clinical practice often separates a
2005;YuandNeimat,2008;Bronsteinetal.,2011).Whilemany patient assessment (“classification”), which translates clinical
patients benefit from DBS, the parameter selection process is dataintoadiagnosis,andatreatmentplan(“controlpolicy”),
largely heuristic, and reprogramming sessions may be weeks or whichtranslatesadiagnosisintoatherapy.Designingthesys-
monthsapart. temtomatchthisseparationenablesphysicianstomoreeasily
Efforthasbeenappliedformorethanadecadetobuildauto- validate and improve algorithms according to their existing
matedsystems(Figure1)thatusepatientstatetoadjuststimula- workflow.
tionparameters,therebyreducingthedelaybetweenstimulation • Partitionscomplexity:algorithmscaninvolvesignificantcom-
updatesbymanyordersofmagnitudecomparedtohumaninter- putationalload,whichisdifficultforimplantablesystemsdue
vention.Realizingthesesystemsrequiresdevelopmentofsensors topowerconstraints(LeeKyongetal.,2012).Partitioningthe
to measure patient data and algorithms to translate the data algorithmshouldenablemoremodulartestingandprototyp-
to the appropriate stimulation parameters (Priori et al., 2012). ing;thisisparticularlyusefulwhenalgorithmcomponentscan
Complexity in the nervous system motivates partitioning the be externalized to allow greater computational freedom than
algorithminto two components: onethat translates sensordata theimplanteddevicecanprovide.Oncevetted,algorithmswith
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|1
Afsharetal. Translationalneuromodulationprototypingplatform
thedesiredtrade-offsbetweenperformance,latency,andpower todevelopnewproblemsorhypothesesregardingthealgorithm.
consumption can be committed to embedded firmware for Iterative testing allows the critic to simultaneously learn about
untetheredoperation. the environment and develop the best performance element to
modulateit.
The “agent-environment” model from artificial intelligence Theagent-environmentmodelissuitableforthedevelopment
research is one model for describing the relationship between ofneuromodulationsystemsforseveralreasons.Themodel:
the physician and the automated neuromodulation system in
learning and implementing algorithms (Figure2). The goal of • Includes the physician-researcher’s involvement to capture
the agent is to develop a performance element (i.e., algorithm) subjectbehaviortovalidatethealgorithm.
to model the relationship between environmental percepts and • Describes the role of the performance element not only as a
actions taken by effectors. The informed critic (i.e., clinician- keyelement ofthe automated closed-loop.system, butalsoas
researcher) updates the performance element by learning from themechanismforthephysician-researchertolearnaboutthe
itsinputdata(sensors)andintermediateprocessing(knowledge) nervoussystem.
• Captures the importance of developing better sensors and
effectors to improve the ability to monitor and modulate the
nervoussystem.
• Captures the iterative learning process needed to develop a
first-principlesunderstandingoftheneurologicaldiseases.
• Leavesthenatureofthealgorithmopen,keepingfreethechoice
ofmachine-learningtechniques(e.g.,supportvectormachine,
Kalmanfilter) and data types (e.g., accelerometer, gyroscope,
biopotential).
The translation of automated closed-loop systems has been
helped by the development of more sophisticated neural sen-
sorsaswellasimprovedunderstandingoftheneuralsignalsthat
underliedisease.Neurochip-2(Zanosetal.,2011)andHermes-D
(Miranda et al., 2010) are two examples of technology to mea-
sure from the network. Neurochip-2 provides three channels of
sensing and stimulation and allows for fast response loop clo-
sure to explore concepts like neural plasticity. The Hermes-D
systemallowsforwireless,largerscalemeasurement(32channels)
of activity, but lacks stimulation capability. Both systems have
FIGURE1|Simplifiedmodelofaclosed-loopneuromodulation
the advantage of higher bandwidth, which allows for measure-
system.
ment ofsingle unit activity, butdraw greater than 1000×more
FIGURE2|Generalizedframeworkforalearningagent(reproducedwithpermission;WikiCommons).
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|2
Afsharetal. Translationalneuromodulationprototypingplatform
power for operation than a typical DBS implant, giving them 2002)havealsobeenfoundinessentialtremor.Synchronization
longevity of at most a few days between recharges. Moreover, in even lower frequencies (alpha and theta range) has been
the limited biomarkers and control variables currently known found in dystonia (Liu et al., 2002; Silberstein et al., 2003;
for neurological diseases motivate the development of platform Kühn et al., 2009; Sharott et al., 2008; Singh et al., 2011).
technologies to enable improved first-principles understanding, Correlations between powerinfrequency bandsaslowasalpha
which may lead to more rapid clinical translation. A critical (Zumstegetal.,2006)andashighas500Hz(Blancoetal.,2011)
step in developing this understanding is the ability to pro- have been reported in patients with epilepsy. Equally impor-
videsimultaneousneuralrecordingandtherapeuticstimulation, tantly, it has been shown that the effect of therapy can be
which is lacking in many research tools today. This capabil- correlated with LFP signals both in DBS (Eusebio et al., 2012;
ity isneeded to understand the system transfer function, which Prioriet al., 2012)and levodopatherapy(Rossiet al., 2008). In
we define as the relationship between stimulation and network aggregate, these studies suggest that LFP is a promising sensor
behavior. input for automated systems treating a variety of neurological
The study of biopotential biomarkers has shown spectral disorders.
powerinlocalfieldpotentials(LFP)tobeadisease-relevantindi- In this work, we describe a platform for investigating these
catorinavarietyofsettings(SchnitzlerandGross,2005;Uhlhaas neuralsignalstowardthedevelopmentofanautomated,closed-
andSinger,2006).Inparticular,thesesignalsareusefulinstudy- loopbioelectronicneuromodulationsystem.Theplatformcom-
ing networks of thalamo-cortical structures and their dynamic prisestoolsandaprocessflowtomapthegenerallearningagent
inter-relationships,whereabnormalneuralsynchronyisbelieved to neuromodulation research and enables rapid prototyping of
tobeahallmarkofdiseasestates(LlinasandRibary,2001;Siegel these tools in an implantable neuromodulation device. We use
et al., 2012). Furthermore, quantified differences in neural syn- a preclinical, in vivo nervous system model to demonstrate the
chrony,whichcanbemeasuredbycalculatingpower(uV/rtHz)2 functional components of the system: collection of neural data,
in aparticularfrequencyband (forexample, “beta”), havebeen identificationofrelevantfeatures(i.e.,biomarkers),development
shown to correlate with symptom severity. For instance, power of the algorithm, and consolidation of the algorithm into an
in the beta band (15–30Hz) has been found to be related to implanteddevice.
cardinal Parkinson’s symptoms such as bradykinesia and rigid-
ity (Hammond et al., 2007; Eusebio and Brown, 2009; Kühn SYSTEMSTRATEGYANDINFRASTRUCTURE
et al.,2009;Priorietal.,2012). Characteristicchanges inpower Toimplementthissystemwemappedthegenerallearningagent
at the theta tremor frequency (Hellwig et al., 2001) and coher- functional blocksinto theneuromodulationdomain(Figure3).
ent activity in the 6–15Hz frequency band (Raethjen et al., Theinterfaceisbi-directional,extractingmeasuresofneuralstate
FIGURE3|Mappingageneralizedlearningmodeltothe componentsthatinterfacewiththeenvironment.Algorithmprototyping
neuromodulationsystem;thecomponentsthatareimplantedare occursintheagentwherethephysicianandresearchercangeneratenew
highlightedbythedashedbox.Theshadedboxesrepresenttheimplanted algorithmsbasedonhistoricaldataandalgorithmperformance.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|3
Afsharetal. Translationalneuromodulationprototypingplatform
through percepts and actuating states in the nervous system prototypeforvalidation:collectionofsensedneuraldata;design
through effectors. Percepts are received through a combination of the performance element’s classifiers based on biomark-
ofsensorsthatincludebioelectricalsensingfromelectrodes(e.g., ers; development of the performance element control policy
ECG, EMG, and LFP) and inertial sensing, (e.g., posture and based on measured neural system identification; and embed-
activity). The effector pathway is defined by electrical stimula- ding of the performance element into the device for chronic
tionpulsepatterns,withparameterssimilartoapprovedtherapy validation.
devices. Todothis,wedesignedasystemwiththefollowingfeatures:
The challenge in designing the performance element is that
characteristics of both percepts and effectors are still evolving. ◦ Implantable device for delivering stimulation including the
The algorithm addresses this ambiguity through use of classi- followingcomponents:
fierportion thatmapssensed signalsto estimates ofstate anda
control-policy portion that maps state estimates into a desired • Bioelectric sensing with 4 bipolar sensing channels with
stimulation. 150nV/rtHz noise floor without stimulation and 300
We have implemented the learning system using an nV/rtHznoisefloorwithstimulation(nb:Stanslaskietal.,
implantable research device and external application tool 2012describesconstraintsofsensingduringstimulation).
coupled with real-time telemetry; the system is illustrated in • Inertialsensingwithacustomthree-axisaccelerometerwith
Figure4. We call this partition of external learning elements a 10mg-rms resolution floor drawing under 600nW/axis
that can be transferred to the implantable device performance (Denisonetal.,2007).
element a “hybrid” design approach. The goal is to construct • Stimulationusingacommerciallyavailableneuralsimulator
a complete platform (combining hardware, software, and systemwithacceptedtherapy.
firmware) for the learning procedure. The learning protocol • Embeddedalgorithmwithindependentlymodifiableclassi-
includes four main steps from initial exploration to a chronic fierandcontrol-policyalgorithms.
FIGURE4|Functionalflowdiagramofthehybridimplantablesystemwiththeinternalandexternalpartitionsdenoted.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|4
Afsharetal. Translationalneuromodulationprototypingplatform
◦ Externaltoolforlearningandprototypingclassifierstotrans- micromachined three-axis accelerometer that transduces capac-
latesensordatatostateestimates. itivefluctuationstoavoltageoutput.Theresolutionfloorofthe
inertialelementis10mgrms,ina20-Hzbandofdetection.The
• Save, parse, and annotate data collected from implantable
sensor draws a total of 2 uW during normal operation, which
device.
minimizeslongevityimpactinthedevice(Denisonetal.,2007).
• Implement,prototype,andcomparemachinelearningalgo-
The sensor inputs from bioelectric and inertial sensors can be
rithms.
fusedtogetherinthealgorithm,ifdesired.
• Developandtestclassifiersfortheimplantedsystem.
Data acquisition also provides an opportunity for optimiz-
◦ Externaltoolforlearningandprototypingcontrolpoliciesto ing efficiency. While the device supports streaming telemetry
translatestatetostimulationupdates. for time and power channels, it is limited to environments in
whichthesubjectisclosetoatelemetrysystem,anddesireddata
• Stream data directly from the implantable device to an
sampling frequency is low. Event triggered recordings allow for
externalprocessorwithlatencylessthan1s(0.5stypical).
timedsegmentsofhighsamplingfrequencydatawhenthesubject
• Send stimulation parameter updates to the implantable
isambulatory.Triggersincludeuserprogrammable,timer-driven
device with latency from command to stimulation at the
intervals;embeddedclassifiers;externalsubjectbuttonpresses;or
electrodeinlessthan1s(0.5stypical).
combinations thereof. For a typical event structure like motion
• Monitor state transitions in classifier and control-policy
orseizureonsets,an8-slooprecordingcouldbeappliedfortwo
algorithms.
recording channels. With a typical data rate of 422Hz, approx-
◦ Telemetry system for retrieving data, modifying classifiers, imately 200 recordings can be stored by the embedded SRAM
prototypingcontrolpolicies,andrewritingdevicefirmware. until it needs to be downloaded and cleared. To organize and
managethe resulting numberoffilesgathered overalongitudi-
The key for this system is to integrate all necessary elements to nal study, a file system wasdeveloped to provide data structure
provide a complete platform for an accelerated learning proce- toresearchers.Informationsuchaseventtimestamps,parameter
dureamenabletorapid-prototypingandclinicaltranslation.The settings, andeventtypeisembeddedinthedataduringrecord-
detailsofthesestepsfollowbelow. ingandautomaticallyextracted asacompanionfileto thedata.
Thecombinationofthecustomintegratedhardware,signalpro-
COLLECTIONOFSENSEDNEURALDATA cessing strategy, and data gathering infrastructure facilitates the
Thedesignoftheperformanceelementstartswithdatacollection. designoftheperformanceelement.
While there are many methods to sense biopotential data, fully
implanted devices offer the advantage of higher signal fidelity LEARNING→PERFORMANCEELEMENTI:CLASSIFICATION
thanfullyexternaldevices(e.g.,EEG),reducedinfectionrisk,and The first subsystem of the performance element is a classifier
improved chronic, ambulatory data collection capability com- to estimate the state of the nervous system from the sensed
pared with implanted devices with external components (e.g., LFP biopotentials. Following the hybrid approach of our plat-
externalizingleadsduringDBSsurgeryortheHermes-Dsystem). form, we implement the classifier as both an internal function
Wehavepreviouslydescribedthedesignandimplementationof of the implantable device and as an external tool for learning
ourfullyimplanted,bi-directionalneuralinterface(Rouseetal., andproblemgeneration;the functional flowofthetool isillus-
2011).Inbrief,thedevicecontainsbothsensingandstimulation tratedinFigure5.Theexternaltoolallowsuserstovisualizetime
components.Thestimulationfeatureembodiesthecapabilityof domain and spectral data, graphically annotate biomarkers of
a commercial DBS system. Biopotential sensing is enabled with interest, and automatically generate classifiers using supervised
acustom-integratedinterfacechipthatallowsformeasurements machine-learning algorithms. In addition, classifier sensitivity
of LFP generated from EMG, ECoG, LFP, and ECG (Avestruz andspecificitycanbeadjustedmanuallytoobtainthedesiredper-
et al., 2008), with noise floor of 150 nV/rtHz without stimula- formance. The resulting classifiers can be stored and compared
tionand300nV/rtHzwithstimulation(nb:Stanslaskietal.,2012, using automatically computed detection statistics. Beyond data
givesdetailsandconstraintsofsensingduringstimulation).The manipulation,thekeyvalueofthetoolisitsrelationshipwiththe
custom integrated circuit (IC) provides data analysis for up to implanteddevice;thetool:
fourbipolarchannels,whichareselectablebetweenNyquist-rate
waveforms(i.e.,timechannels)andspectralpoweratspecificfre- 1. Servesasadatarepositoryforgroupingandsortingdatafiles
quencybandsofinterest(i.e.,powerchannels).Thetimechannels fromdifferentrecordingsessions.
provide complete spectral information; however, they incur the 2. Parses data collected from the implanted device, automati-
penaltyofmuchhigherpowerconsumption.Powerchannels,on cally accounting for differences in formatting and recording
the other hand, extract a powerenvelope that isdown-sampled settings.
to 5Hz prior to digital signal processing. The reduction of sig- 3. Creates algorithms that can be uploaded directly into the
nal dynamic rangeprior to digitization is a common technique implantabledevice.
for saving energy in micropower systems. The design model is
tousethetimechannelsforneuralsystemidentification,includ- Thedefaulton-boardclassifieralgorithmisalinear-discriminant
ingidentifyingbiomarkersandtotransfertothepowerchannels using a modified Fischer-discriminant approach; it is a linear
to optimize efficiency chronically. The inertial element uses a decisionboundaryinauser-selectablefeaturespacethatidentifies
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|5
Afsharetal. Translationalneuromodulationprototypingplatform
FIGURE5|Functionalflowfordataannotationandclassificationusingtheexternalsoftwaretool.
an event signal sample from other samples. The algorithm was devicesensesandsavesbiopotentialdatatotheinternalmemory.
designedusingreducedsetmethodsasdescribedin(Shoebetal., Once retrieved from the device, system identification is per-
2009). The use of the multi-dimensional linear boundary was formed by measuring the relationship between the stimulation
foundtooptimizetrade-offsinpowerconsumption,latency,sen- parametersandbiopotentials.
sitivity, and specificity. Recent work by Lee describes a similar Second,the control policymaybeadjusted inreal-time ona
trade-offcalculationandsupportsourdesignchoices(LeeKyong researcher’s device using an external device to wirelessly trans-
et al., 2012). The on-board algorithmcanbe used fordetecting fer data: sensed data is passed to the researcher’s device and
events, which are time-stamped and used to trigger recordings control-policy output is passed to the implantable device. This
while the subject is ambulatory, thereby reducing current drain capability enables prototyping algorithms including the use of
nearly 100-fold and reducing classification latency 5-fold, from tapped-delaylinesandtimesynchronizingwithothersensorsand
∼1s to ∼200ms. If the biomarker’s characteristics warrant a hardware, and deriving a variety of signal features (e.g., phase
morecomplexclassifierorshorterlatency, thealgorithmcanbe amplitude coupling). The external device ensures data integrity
updated,tradingoffpowerconsumption. inbothdirectionsthroughcyclic-redundancychecksandensures
patient safety by returning the device to safe, pre-programmed
LEARNING→PERFORMANCEELEMENTII:CONTROLPOLICY stimulation state should the researcher’s control policy behave
Thesecondalgorithmsubsystemisthecontrolpolicythatmaps unexpectedly.Additionalsafetyisensuredbyallowingthecontrol
the state estimate into an optimal stimulation sequence. Like policy to select only among stimulation parameter boundaries
theclassifieralgorithm,weimplementedthecontrol-policyalgo- thathavebeenpredeterminedbytheresearcher.
rithm both internal to the device and as an external system For the platform design, particular attention was paid to
forlearningandproblemgeneration.Non-linearitiesinnetwork the latency in the telemetry links, which is a key factor to
dynamicsheightentheneedtosamplemanyinput–outputpairs effectively study the dynamics. In the first generation of devel-
forsystemidentification.Thiscanbeaccomplishedintwoways: opment, we required that total latency through the channel
First,theexternaltoolmaybeusedtosweepanystimulation be constrained to 1s or less, and typically under 0.5s. This
parameter (e.g., amplitude or frequency) while the implantable degreeoflatencyissuitableformanyclosed-loopalgorithmsthat
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|6
Afsharetal. Translationalneuromodulationprototypingplatform
operate on timescales of seconds, hours, or days. The inherent wasonexploringthebioelectricalpropertiesofthenetworkand
latency of the links was dominated by two factors: the first is building up a closed-loop system; the conceptual schema for
the data packet format and error correction handshakes using developinginertial-basedsystems,classifiersandcontrolpolicies
the 175kHz ISM band, and the second is the internal packet waspreviouslydemonstratedwiththisarchitecture(Schultzetal.,
transferwithinthebioelectronicdevice,which,forsafetyreasons, 2012).
are secondary interrupt priorities compared to the therapeu-
tic stimulation. Although the latency can be much improved PHYSIOLOGICALMETHODS
by running in the device, it limits flexibility during the initial The in vivo device was chronically implanted in an ovine
learning phase. Therefore, for most cases, the new stimulation animal model conducted under an IACUC-approved protocol
parameters are generated externally, where algorithms can be (Stypulkowski et al., 2011) and is summarized here. Following
made arbitrarily complex and rapidly evaluated to see if they anesthesia, 1.5TMRIswerecollected andtransferred toasurgi-
capture the desired behavior of the neural system. It is highly calplanningstation.Trajectoriesforaunilateralanteriornucleus
desirableto validate the behaviorpriorto committing to verifi- (AN)DBSlead(Medtronicmodel3389)andunilateralHClead
cationofembedded firmwaredueto regulatory constraints and (Medtronicmodel3387)wereplanned,andleadsimplantedusing
requirements. For example, the platform can implement arbi- aframelessstereotacticsystem(NexFramefromMedtronic,Inc.).
trary control paradigms such as simple bang–bang controllers Once lead placement was confirmed based upon electrophysio-
(modeledfromearlycardiacdefibrillators)ormoresophisticated logical measures, Medtronic model 37083 extensions were con-
proportional-integral-derivative and linear-quadratic-Gaussian nectedtotheDBSleads,tunneledtoapost-scapularpocket,and
controllers for achieving the optimal path to the desired state connectedtotheprototypechronicimplantabledevice.Figure6
maintenance. illustratestheoverallsystemplacementandsetup.Followingclo-
sureofallincisions,anesthesiawasdiscontinued,andtheanimal
COMMITTINGTHEPERFORMANCEELEMENTTOTHEEMBEDDED wastransferredtosurgicalrecovery.
DEVICEFORVALIDATION Allsensingandstimulationdocumentedherewereconducted
After learning and prototyping the classifier and control pol- inasingle,awakesheeprestinginasling.Inthisparticularwork,
icy, the algorithm can be validated by embedding onto the allreporteddatawererecordedfromtheHCwithbipolarmon-
implantabledevicefirmwareusingtelemetry. Thefirmwareuses tageusingcontactssurroundingamonopolarstimulationcontact
a dedicated boot loader that allows for a new series of code (square,biphasic300µspulsewidthonE1withfar-fieldreturn)
to be flashed to non-volatile memory inside the device in a to mitigate artifacts via common-mode rejection during stim-
few minutes. The firmware in the device is partitioned such ulation (Stanslaski et al., 2012); functional network data from
that the classifier and control policy can be updated indepen- thalamicstimulationandsensingarenotshown,butcanbefound
dently of the therapy code, thereby keeping the interaction in Stypulkowski et al. (2011). Neural data, stimulus trains and
to that necessary for real-time classification and closed-loop classifierdetections wererecordedandsavedbyPCsoftwarevia
operation. To assist in validation, the firmware is capable of wirelesstelemetry. Dataweregatheredover15monthsandrep-
streaming out the classifier and control-policy states in addi- resents over18months ofoperationwiththedevicecompletely
tion to sensed signals in real-time, so that the user has visi- implanted.
bility into the algorithm operation. For chronic operation, the As background to the analysis that follows, our physiologi-
state transition information is included in the data log for cal system relies on three qualitatively discernible states in the
validation. biologicalsystem:
METHODS:DEMONSTRATIONOFTHELEARNINGAGENT 1. Resting: defined as the state before any stimulation/
ARCHITECTURE neuromodulationhasoccurred.
As demonstration of the capabilities of our method and tools, 2. After-discharge (AD):definedasthestateofhigh-energyLFP,
we used the system to investigate, characterize and dynamically similar to a seizure event, and by characteristic head move-
modulatethehippocampaldynamicswithinthecircuitofPapez. mentsofthesubject.Inourdefinition,theADcouldoccurat
The circuit of Papez is a thalamo-cortical circuit implicated in anytime,independentofstimulationdelivery.
temporal lobe epilepsy and involves a reentrant loop involving 3. Suppression:definedasthestatewithactivitythatisbelowthe
the hippocampus (HC) and thalamus. The goal was to design nominalrestingstate.
from first principles a demonstrative “homeostatic” feedback
loop, which would titrate stimulation dynamically to maintain LEARNINGFLOWMETHODOLOGY
network activity reflected in the field potentials; the intention The system was deployed on the physiological preparation to
was to show the capabilities of the technology, as opposed to developanembeddedclosed-loopalgorithmusingourtoolsand
demonstrateorclaimatherapeuticalgorithmperse.Designofthe processes.Thetechnicalmethodsappliedthedesignflowoutlined
loop required that we address many issues of neuromodulation inthesystemarchitecturetothephysiologicalpreparation:
design:testinginanawakeandfreelymovingsubject,considera-
tionforreliabilityandrepeatability,andchronicimplantstability • Collectionofsensedneuraldata
and safety. Methods are detailed from the physiological prepa- Using the bi-directional telemetry link and embedded data
ration and technology points of view. The focus of this effort gathering capabilities, we gathered baseline training data on
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|7
Afsharetal. Translationalneuromodulationprototypingplatform
FIGURE6|Closed-loopneuromodulationsystemimplantedinanovinemodel.ThefigureisreproducedfromStanslaskietal.(2012)withpermissions
fromtheIEEE.
background network activity. We also used the stimulator withincreasingstimulationamplitude,suggestingsuppressionof
and sensing functionality to identify useful biomarkers and activity, at least locally to the HC (Figure8, right). To deter-
understandsystemtransferfunctions requiredforclosingthe mine whether the network wastruly suppressed, we performed
feedbackloop. asecondseriesoftransferfunctionexperimentswhichmeasured
• Learning → Performance element I: design of classifier thepre-stimulationbaselinebetabandpowerlevelfollowedwith
algorithm a high amplitude delivery (≥1.50V) “probe pulse” capable of
Thesoftwarealgorithmtoolwasusedtodevelopclassifiersto inducingAD.Becauseourexperimental setup wasnotaseizure
support the after-discharge detection and verify suppression model,weusedthepost-stimulationADdurationasthedesired
levels,whichwerevalidatedusingthereal-timetelemetrylink. outputforassessmentofnetwork effect. Throughspectral anal-
• Learning → Performance element II: development of the ysis of the data, we observed a potential control variable in the
controlpolicy 20±2.5Hz band (approximately the beta band) that seemed
Afterdevelopmentoftheclassifiers,theauto-detectionofafter- correlatedtothequalitativelyobservedstates:
discharges and therapy titration was validated using off-line,
real-timeprocessingwiththebi-directionaltelemetrylink.Key
1. Resting state corresponded to relatively constant beta band
parameterswereverifiedtobeacceptablefortiminglatency.An
power(approximately2.7uVrms).
additionalalgorithm(datanotshown)wastestedtoshowthe
2. ADstatecorrespondedtoincreasedbetabandpower(approx-
systemcouldautomaticallysearchtheparameterspacetofind
imately30uVrms).
acceptablesuppressionbehavior.
• Committing the performance element to the embedded 3. Suppressionstatecorrespondedtodecreasedbetabandpower
(approximately1.5uVrms).
deviceforvalidation
The final embedded algorithm implemented three sub-
algorithmsintoasingle-statemachine:ADdetectionandmit- We characterized the biomarker over 15 months of data collec-
igation;suppressiondetector;andparametersearch.Thecode tion. For data shown, the units of spectral power in all data
was then downloaded to the device through wireless teleme- figuresare(uV/rtHz)2,withanarbitraryscalereferredtoasleast
try, error checked for complete flash writes, and the implant significant bit (LSB). Results showed that AD generation was
was then activated with the closed-loop algorithm. All states
a probabilistic function of stimulation amplitude; stimulation
wereexercised inthealgorithmroutinetovalidateoperation. below 1.5V did not result in any AD, stimulation between 1.5
Statetransitionswerealsorecordedinthedevicedatarecords
and1.7VresultedinoccasionalADs,andstimulationabove1.7V
for automated annotation of files, allowing for observational
alwaysresultedinADs(datanotshown).Furthermore,ADdura-
validationandalgorithmrefinement.
tionappearedtobeafunctionofthebetabandpre-stimulation
state;thegreaterthepre-stimulationbetabandpowerabovethe
RESULTS definedsuppressedstate,thegreatertheADduration(Figure7).
COLLECTIONOFSENSEDNEURALDATA:IDENTIFICATION Furthermore,theseobservationswererobust:thesuppressedstate
OFBIOMARKERSANDALGORITHMS betabandpowervariedbylessthan2LSBovertheentireduration
We aimed to explore the states of the system to find of the experiment. These results imply that spectral beta band
relevant control-variable biomarkers in vivo. Analysis of the power could be a control variable of interest when modulating
post-stimulationdatashoweddecreasingmeanbetabandpower networkstate.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|8
Afsharetal. Translationalneuromodulationprototypingplatform
To further understand the dynamic state of the system, we Thetitrationsweepfordeterminingnetwork-stateresponseto
aimedtocharacterizethetransferfunctionbetweenourproposed stimulationisacriticalstepindesigningtheneuralcontrolalgo-
biomarker—spectralbetabandpower—andstimulationpatterns. rithm.Thedatasuggestthatstimulationcanhavedifferenteffects
To characterize the response of the biomarker to stimulation, onthenetwork: whilelowandmoderatestimulationamplitude
we ranseveraltitration sweeps. Therecorded biomarkersignals appears to suppress the network excitability, high stimulation
were capturedatrest, duringAD events, and duringdeliveryof amplitudecaninduceanAD.Basedontheseresults,wewanted
stimulation at several amplitudes (0.75–1.7V) and frequencies touseourplatformtoimplementaperformanceelementtohave
(50,120Hz)inordertodetermineareferencevaluetodiscrim- two key features: (1) change stimulation amplitude to keep the
inate both suppression and ADs. Figure8 shows the network networkatthebalancepointofsuppressionandinductionofAD
responseduringstimulation(25s,red)andbetweenstimulation and(2)duetotheprobabilisticnatureofADinduction,allowfor
periods(25s,blue).Importantly, thedetection ofADinduction thedetectionofADinreal-timetoabortstimulationandadjust
required sensing neural activity in the presence of stimulation the stimulation levels lower. To do this, we designed the per-
(Stanslaski et al., 2012) and would have been lost if channel formance element in two parts: states classification and control
blankingwereemployed. policyimplementation.
LEARNINGPERFORMANCEELEMENTI:DESIGNOFCLASSIFIER
ALGORITHMS
To automate a control loop, we used the observed qualitative
correlations with a quantitative algorithm to detect the AD in
real-timewithaclassifierconstructedwiththeexternalclassifier
tool. To help mitigate stimulation artifact, we also used spec-
tral band (approximately 70Hz) to capture stimulation energy
inthenetworkwithoutbeingconfoundedbyobservablechanges
in neural physiology. To achieve this, we applied a measure
of stimulation artifact as a feature input within the algorithm
to distinguish stimulation result and non-stimulation result as
described in Stanslaski et al. (2012). We include the two power
channel outputsinFigure9fordemonstrationpurposes,show-
ing correlation between the amplitude of beta band power and
ADinFigures9A,B.
After annotation was supplied to the training data sets, we
FIGURE7|After-dischargedurationasafunctionofbetaband used the tool to develop a linear, binary classifier to detect AD
powerincreasefromsuppressedbaseline.Highamplitudestimulation with and without stimulation. The detection probability den-
parameterswerekeptconstantinagivensessionandwerealways sity plot, receiver operating characteristic (ROC) curves, and
determinedtobesufficienttoinitiateanAD.
detection cross-validation result, which are directly generated
FIGURE8|Determination of the hippocampal network transfer amplitudes, followed by an increase in beta band power at higher
function between stimulation and beta band spectral power. stimulation amplitudes, resulting in occasional AD during stimulation
There is an initial reduction in beta band power at low stimulation at 1.5V.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|9
Afsharetal. Translationalneuromodulationprototypingplatform
FIGURE9|Training the classifier to detect the onset of of detection states versus distance from the classifier boundary.
after-discharges in the presence of stimulation. (A) Is a dataset with (D) Estimates the true positive (TP) and false positive (FP) percentages
a representative AD. “Discard transient” periods refer to portions of based on the classifier. (E) Shows the impact of onset and termination
the signal that were not used in training. (B) Is a dataset for providing constraint logic on detector specificity by overlaying the estimated
stimulation artifact without an AD present. (C) Provides the histogram detector state with recorded data files.
by the software tool, are presented in Figures9C,D, and E, the neural system could be quantitatively classified on-line as
respectively. The detection probabilityhistogram (C) represents suppression,AD,orresting.
themagnitudeofthestatefromtheboundary,allowingformul-
tipledimensionsofdatatocollapsetoasinglegraphbiomarker LEARNINGPERFORMANCEELEMENTII:DEVELOPMENTOFTHE
separation. The detection probabilities graph (D) provides an CONTROLPOLICY
estimate of the true-positive and false-positive rates based on With the classifier in place, we next determined the control
the derived classifier.The filtered detection summary graph(E) policy. Given the unknown neural dynamic requirements and
allows for the user to set onset and termination duration con- algorithm parameters, the control policy was first prototyped
straints (i.e., a minimum duration in a classified state before using the hybrid development partition to determine the stim-
detectionisdetermined)tohelpimprovespecificityattheexpense ulation amplitudes and changes that would be used for each
of classifier latency. Graph (E) shows an overlay of the clas- state. Figure10 illustrates an example of this testing to show
sification state over the data. We downloaded and embedded that stimulationcaninduceboththe ADstate andthesuppres-
intotheimplanteddevicetheclassifierthatoptimizedsensitivity, sion state. In this test, the controller logic uses two stimula-
specificity,andlatencytrade-offs. tion programs.In the cyclestim (CS) program,high amplitude
In addition, we used the tool to develop a separate classifier stimulation (1.50V) capable of inducing an AD is cycled on
thatcoulddetect thepresenceofthesuppressionstatebasedon and off, while spectral power in critical bands and classifier
the beta signal. This was also tested and similarly embedded in state is continuously telemetered out of the device. If the clas-
the implanted device. Thus, with these classifiers, the state of sifier does not detect an AD, stimulation continues to cycle.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article117|10