BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20240626T180034Z
LOCATION:3003\, 3rd Floor
DTSTART;TZID=America/Los_Angeles:20240626T141500
DTEND;TZID=America/Los_Angeles:20240626T143000
UID:dac_DAC 2024_sess125_RESEARCH1842@linklings.com
SUMMARY:On the Design of Novel Attention Mechanism for Enhanced Efficiency
  of Transformers
DESCRIPTION:Research Manuscript\n\nSumit Jha (Florida International Univer
 sity), Susmit Jha (SRI International), and Rickard Ewetz and Alvaro Velasq
 uez (University of Central Florida)\n\nWe present a new XOR-based attentio
 n function for efficient hardware implementation of transformers. While st
 andard attention relies on matrix multiplication, we propose replacing the
  computation of this attention function with bitwise XOR operations. We ma
 thematically analyze the information-theoretic properties of multiplicatio
 n-based attention, demonstrating that it preserves input entropy, and then
  show that XOR-based attention approximately preserves the entropy of its 
 input. Across various simple tasks, including arithmetic, sorting, transla
 tion, and text generation, we show comparable performance to baseline meth
 ods using scaled GPT models. XOR-based attention shows substantial improve
 ment in power, latency, and area compared to the multiplication-based atte
 ntion function.\n\nTopic: AI, Design\n\nKeyword: AI/ML System and Platform
  Design\n\nSession Chairs: Amin Firoozshahian (Rain AI) and Thierry Tambe 
 (Stanford University)
END:VEVENT
END:VCALENDAR