# Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization Outline Thank you !

## Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization Outline Thank you !

Dunn, Heather, Executive Producer has reference to this Academic Journal, PHwiki organized this Journal Delay Reduction via Lagrange Multipliers in Stochastic Network Optimization Longbo Huang Michael J. Neely EE@USC WiOpt 2009 Sponsored in part by NSF Career CCF in addition to DARPA IT-MANET Program Problem as long as mulation Backlog behavior under Quadratic Lyapunov function based Algorithm (QLA): an example General backlog behavior result of QLA as long as general SNO problems The Fast-QLA algorithm (FQLA) Simulation results Summary Outline Problem Description: A Network of r Queues Slotted Time, t=0,1,2, S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, r in addition to om arrivals, etc.) x(t) = Control Action, chosen in some abstract set X(S(t)) (e.g. power/b in addition to width allocation, routing) (S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: j(t)=bj(S(t), x(t)) packets in queue j The stochastic problem: minimize: time average cost subject to: queue stability. [f(), g(), b() are only assumed to be non-negative, continuous, bounded]

This Particular University is Related to this Particular Journal

Slotted Time, t=0,1,2, S(t) = Network State, Time-Varying, IID over slots (e.g. channel conditions, r in addition to om arrivals, etc.) x(t) = Control Action, chosen in some abstract set X(S(t)) (e.g. power/b in addition to width allocation, routing) (S(t), x(t)) costs: f(t)=f(S(t), x(t)) generates: Aj(t)=gj(S(t), x(t)) packets to queue j serves: j(t)=bj(S(t), x(t)) packets in queue j The stochastic problem: minimize: time average cost subject to: queue stability. QLA achieves: [G-N-T FnT 06] Avg. cost: fav <= fav + O(1/V) Avg. Backlog: Uav <= O(V) [f(), g(), b() are only assumed to be non-negative, continuous, bounded] Problem Description: A Network of r Queues An Energy Minimization Example: The QLA algorithm U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) The QLA algorithm (built on Backpressure): 1. Compute the differentiable backlog Wii+1(t)=max[Ui(t)-Ui+1(t), 0], 2. Choose (P1(t), P5(t) that maximizes: i[Wii+1(t)i(Pi(t)) -VPi(t)] =i[Wii+1(t) Si(t) V]Pi(t) e.g., if S2(t)=2, then if W23(t)2>V, we set P2(t)=1. Link 2->3 Goal: allocate power to support the flow with min avg. energy expenditure, i.e.: Min: avg. iPi s.t. Queue stability W23(t) U2 U3 An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) Queue snapshot under QLA with V=100, first 100 slots: U1 U2 U3 U4 U5 Goal: Min: avg. iPi s.t. Queue stability size time

An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) Queue snapshot under QLA with V=100, first 500 slots: U1 U2 U3 U4 U5 size time Goal: Min: avg. iPi s.t. Queue stability An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) Queue snapshot under QLA with V=100, first 1000 slots: U1 U2 U3 U4 U5 size time Goal: Min: avg. iPi s.t. Queue stability An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) Queue snapshot under QLA with V=100, first 5000 slots: U1 U2 U3 U4 U5 size time Goal: Min: avg. iPi s.t. Queue stability

An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) Queue snapshot under QLA with V=100, (U1(t), U2(t)): (500,400) t=1:500k Goal: Min: avg. iPi s.t. Queue stability An Energy Minimization Example: Backlog under QLA U1 U2 S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 R(t) 1(t) 2(t) 3(t) 4(t) 5(t) t=1:500k t=5k:500k Queue snapshot under QLA with V=100, (U1(t), U2(t)): (500,400) Goal: Min: avg. iPi s.t. Queue stability General result: Backlog under QLA Implications: (1) Delay under QLA is (V), not just O(V); (2) The network stores a backlog vector UV. Theorem 1: If q(U) satisfies C1: as long as some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV = (V).

General result: Backlog under QLA Implications: (1) Delay under QLA is (V), not just O(V); (2) The network stores a backlog vector UV. Lets subtract out UV from the network! Replace most of the UV data with Place-Holder bits Theorem 1: If q(U) satisfies C1: as long as some L>0 independent of V, then under QLA, in steady state, U(t) is mostly within O(log(V)) distance from UV = (V). Fast-QLA (FQLA): Using place-holder bits A single queue example: Start here First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q as long as all t>=t0. (2) Let U(0)=Q, run QLA. Fast-QLA (FQLA): Using place-holder bits A single queue example: Start here reduced Advantage: delay reduced by Q, same utility per as long as mance. First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q as long as all t>=t0. (2) Let U(0)=Q, run QLA. actual backlog

Fast-QLA (FQLA): Using place-holder bits A single queue example: Start here actual backlog (V) Advantage: delay reduced by Q, same utility per as long as mance. Problem: Q UV-(V), delay (V). reduced First idea: (1) choose number of place-holder bits Q, s.t., if U(t0)>=Q, then U(t)>=Q as long as all t>=t0. (2) Let U(0)=Q, run QLA. Fast-QLA (FQLA): Using place-holder bits A single queue example: FQLA idea: Choose of place-holder bits Q such that backlog under QLA rarely goes below Q. Problem: (1) U(t) will eventually get below Q, what to do (2) How to ensure utility per as long as mance Fast-QLA (FQLA): Using place-holder bits A single queue example: FQLA idea: Choose of place-holder bits Q such that backlog under QLA rarely goes below Q. Problem: (1) U(t) will eventually get below Q, what to do (2) How to ensure utility per as long as mance Answer: use virtual backlog process W(t) + careful pkt dropping

Fast-QLA (FQLA): Using place-holder bits A single queue example: FQLA: (1) Choose of place-holder bits Q such that backlog under QLA rarely goes below Q. (2) Use a virtual backlog process W(t) with W(0)=Q to track the backlog that should have been generated by QLA. (3) Obtain action by running QLA based on W(t), modify the action carefully. Fast-QLA (FQLA): Using place-holder bits A single queue example: Modifying the action If W(t)>=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)

Fast-QLA (FQLA): Using place-holder bits A single queue example: Modifying the action If W(t)>=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)=Q, same as QLA, admit A(t), serve (t), i.e., FQLA=QLA. If W(t)Q1=4952 & W2(t)>Q2=3952 Simulation U1 U2 R(t) S1(t) S2(t) S3(t) S4(t) S5(t) U3 U4 U5 Backlog % of pkt dropped Quick comparison V=1000, U QLA15V=15000 U FQLA5log2(V)=250 60 times better! Simulation parameters: V= 50, 100, 200, 500, 1000, 2000, Each with 5×106 slots, UV=(5V, 4V, 3V, 2V, V)T.

Summary Under QLA, the backlog vector usually stays close to an attractor  the optimal Lagrange multiplier UV. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay. Summary Note: (1) Theorem 1 also holds when S(t) is Markovian, (2) FQLA-General as long as the case where UV is not known, per as long as mance similar to FQLA-Ideal, (3) when q0(U) is smooth, we prove O(sqrt{V}) deviation bound, (4) The Network Gravity role of Lagrange multiplier. Details see ArXiv report 0904.3795 Under QLA, the backlog vector usually stays close to an attractor  the optimal Lagrange multiplier UV. FQLA subtracts out the Lagrange multiplier from the system induced by QLA by using place-holder bits to reduce delay. Thank you ! Questions or Comments

## Dunn, Heather Executive Producer

Dunn, Heather is from United States and they belong to KPNX-TV and they are from  Phoenix, United States got related to this Particular Journal. and Dunn, Heather deal with the subjects like Local News; National News; Regional News

## Journal Ratings by Ndejje University

This Particular Journal got reviewed and rated by Ndejje University and short form of this particular Institution is UG and gave this Journal an Excellent Rating.