This page shows visualizations of some width-5 1-d convolutional filters from Google's lm_1b language model. Each column corresponds to one position in the filter, and shows the characters with the most positive weights. Use the checkbox in the bottom-right to also see the most negative weights (may be slow).
Below that are examples of words for which the filter emits the highest values. A filter's response is its maximum value over all substrings it sees in the word. So if a filter has high weights on 'c' in the first position, then 'a', then 't', it will assign equally high scores to 'cat', 'fatcat', 'concatenate', etc. The portion of the string in blue is the substring the filter is responding to.
'^' and '$' represent beginning and end of word markers, respectively. '_' is a padding character. Literal versions of those characters are escaped with a backslash.
Use the links at the top to see filters of other widths.
Check out my blog post here for a bit more context.
Show most negative weights
Filter 0 (bias = -0.45) #
<BOS>
g
Q
z
N
U
X
c
"
A
M
d
e
l
W
i
9
G
f
C
F
p
8
s
6
h
q
m
^
u
7
y
\194
T
2
D
3
a
Z
L
j
k
-
f
7
p
J
B
g
t
Y
y
4
K
6
a
5
m
G
c
0
,
(
U
L
T
Z
r
\194
P
3
;
:
A
8
w
9
q
X
O
A
f
Z
v
X
u
G
-
L
s
g
t
Q
d
Y
x
V
k
/
e
C
E
7
F
5
'
K
T
1
o
l
j
n
i
H
U
2
M
"
p
U
x
.
S
u
6
A
5
t
i
r
J
Z
Y
n
0
w
7
H
W
c
3
C
X
a
9
N
o
D
l
m
4
y
G
s
V
_
O
I
n
L
t
U
\194
y
W
G
I
r
v
b
w
E
i
g
M
s
$
a
X
h
,
A
5
d
6
u
-
z
q
\163
'
.
/
P
j
m
B
F
Non-zero for 14.2% of words.
^LAS$ ____(2.5) ^VEGAS$ ____ (2.5) ^acknowledgin g$____ (2.4) ^pledgin g$____ (2.4) ^regain $____ (2.2) ^regain ed$____ (2.2) ^fugit ive$____ (2.2) ^integrit y$____ (2.2) ^GAO$ ____(2.2) ^sellin g$____ (2.1) ^tellin g$____ (2.1) ^travellin g$____ (2.1) ^compellin g$____ (2.1) ^best-sellin g$____ (2.1) ^satellit e$____ (2.1) ^satellit es$____ (2.1) ^reunit ed$____ (2.0) ^three-mon th$____ (2.0) ^DAX$ ____(2.0) ^Egypt $____ (2.0) ^Egypt ian$____ (2.0) ^lapt op$____(2.0) ^lapt ops$____(2.0) ^Mellon $____ (2.0) ^750$ ____(2.0) ^three-poi nt$____ (1.9) ^gap$ ____(1.9) ^750, 000$____(1.9) ^Oscar-win ning$____ (1.9) ^relax$ ____ (1.9) ^gain $____(1.8) ^gain s$____(1.8) ^gain ed$____(1.8) ^gain ing$____(1.8) ^half-cen tury$____ (1.8) ^helpin g$____ (1.7) ^Gap$ ____(1.7) ^7.6$ ____(1.7) ^700$ ____(1.7) ^450$ ____(1.7) Filter 1 (bias = -0.47) #
.
E
n
k
d
S
-
f
Z
B
\194
O
/
K
C
F
D
U
c
e
7
T
X
s
l
r
1
i
q
t
g
P
L
b
'
R
6
p
^
M
S
u
f
d
K
D
O
v
V
.
;
c
W
q
i
a
,
?
M
T
F
-
3
z
X
0
Q
n
%
g
4
\195
5
U
p
1
s
A
Y
)
W
u
E
l
w
h
Q
d
2
p
X
n
K
D
S
r
"
P
5
-
4
T
O
k
Z
m
3
C
e
v
N
R
$
H
8
o
G
j
i
D
m
d
b
I
f
C
o
7
x
1
w
2
-
Q
K
0
s
6
Y
8
l
\194
i
T
y
c
r
F
h
X
k
t
L
9
'
5
G
4
S
Q
p
U
l
V
t
"
o
R
D
Z
h
s
d
$
y
'
q
u
T
Y
e
\194
g
9
i
M
c
b
A
n
r
1
a
O
Non-zero for 15.1% of words.
^SEC$ ____(3.7) ^denied$ ____ (2.8) ^accompanied$ ____ (2.8) ^licensed$ ____ (2.6) ^died$ ____ (2.6) ^studied$ ____ (2.6) ^birdied$ ____ (2.6) ^engulfed$ ____ (2.6) ^SEOU L$____(2.5) ^fed$ ____(2.4) ^linked$ ____ (2.4) ^ranked$ ____ (2.4) ^thanked$ ____ (2.4) ^top-ranked$ ____ (2.4) ^needs $____ (2.4) ^Set$ ____(2.3) ^Secu rity$____(2.2) ^Secu rities$____(2.2) ^need$ ____ (2.2) ^newes t$____ (2.1) ^applied$ ____ (2.1) ^rallied$ ____ (2.1) ^replied$ ____ (2.1) ^supplied$ ____ (2.1) ^diets $____ (2.0) ^Swis s$____(2.0) ^wanted$ ____ (2.0) ^presented$ ____ (2.0) ^appointed$ ____ (2.0) ^pointed$ ____ (2.0) ^granted$ ____ (2.0) ^represented$ ____ (2.0) ^disappointed$ ____ (2.0) ^unprecedented$ ____ (2.0) ^indeed$ ____ (1.9) ^Indeed$ ____ (1.9) ^HSBC$ ____ (1.9) ^communist$ ____ (1.9) ^Communist$ ____ (1.9) ^columnist$ ____ (1.9) Filter 2 (bias = -0.61) #
A
v
.
f
/
p
Q
k
W
P
L
j
Y
J
H
-
N
x
Z
F
y
d
O
i
^
b
X
e
0
B
T
R
9
n
J
y
v
A
9
.
0
!
R
?
j
m
;
h
T
H
P
L
\194
a
B
U
-
s
X
g
6
n
7
\$
\195
/
I
(
V
Z
)
u
E
F
G
.
C
v
S
u
Q
q
p
B
Y
w
"
N
O
-
V
e
$
l
'
J
c
L
y
f
g
a
s
n
k
b
P
t
8
\195
A
o
.
p
v
E
\194
O
D
r
u
i
c
e
C
g
L
P
n
G
/
S
d
3
m
w
U
I
B
2
'
J
l
j
Z
K
z
f
t
\163
T
W
x
w
f
I
9
t
8
g
\194
A
b
O
F
E
6
z
h
p
"
T
v
r
V
i
Y
G
B
c
u
2
'
\163
L
e
M
_
7
a
m
D
Non-zero for 16.0% of words.
^Aviv$ ____ (5.6) ^vs.$ ____(5.0) ^Jr.$ ____(4.3) ^Jimm y$____(3.9) ^Abramovich $____ (3.8) ^Jim$ ____(3.7) ^FARC$_ ___ (3.6) ^viab le$____(3.6) ^viab ility$____(3.6) ^vivi d$____(3.6) ^Avenu e$____ (3.5) ^LPGA$ ____ (3.5) ^Gavin$ ____ (3.4) ^David$ ____ (3.3) ^OECD$ ____ (3.3) ^Djokovic$ ____ (3.3) ^Jankovic$ ____ (3.3) ^Ivanovic$ ____ (3.3) ^Rich ard$____(3.2) ^Rich ardson$____(3.2) ^Rich ards$____(3.2) ^Rich mond$____(3.2) ^Rich $____(3.2) ^visu al$____(3.2) ^vice $____(3.2) ^vice -president$____(3.2) ^Davids on$____ (3.1) ^Jill $____(3.1) ^PGA$ ____(3.1) ^ABC$_ ___ (3.1) ^JPMo rgan$____(3.1) ^vill age$____(3.1) ^vill ages$____(3.1) ^vill agers$____(3.1) ^vill a$____(3.1) ^Mervyn$ ____ (3.1) ^ICC$ ____(3.0) ^Davyde nko$____ (3.0) ^vici ous$____(3.0) ^lavish $____ (3.0) Filter 3 (bias = -0.48) #
A
-
t
P
N
m
.
b
S
f
/
J
Q
u
W
p
O
k
,
v
c
d
5
i
\194
\195
I
V
C
j
o
M
^
U
w
x
a
'
R
R
t
J
p
9
F
N
g
\195
h
Y
y
Z
d
-
i
b
C
K
c
&
S
u
A
"
j
o
e
Q
D
B
,
U
l
:
\$
z
f
?
m
z
f
R
y
U
h
G
m
E
p
Q
n
s
t
9
F
Y
H
I
q
J
e
"
x
2
M
8
i
$
k
0
j
Z
r
7
.
O
l
\194
,
y
b
M
k
,
g
W
z
K
R
D
r
f
.
6
V
O
Y
5
A
2
l
e
\195
3
s
1
a
N
v
4
-
F
'
/
u
H
U
8
G
Q
m
d
k
7
B
2
T
8
b
I
z
1
t
6
g
X
v
3
l
"
A
9
c
4
o
W
M
H
S
q
Y
$
s
/
f
N
G
5
U
Non-zero for 15.9% of words.
^JERUSALEM$ ____ (3.3) ^PARIS$ ____ (3.2) ^KABUL$ ____ (2.9) ^AZUZ$ ____ (2.9) ^accused $____ (2.9) ^focused $____ (2.9) ^Arsen al$____ (2.7) ^Arsen e$____ (2.7) ^housed $____ (2.7) ^used $____(2.6) ^caused $____ (2.5) ^fantasy$ ____ (2.5) ^Austr alia$____ (2.5) ^Austr alian$____ (2.5) ^Austr ia$____ (2.5) ^Austr ian$____ (2.5) ^ANGEL ES$____ (2.4) ^ISLAMABAD$ ____ (2.3) ^NEW$ ____(2.3) ^unused $____ (2.3) ^closed $____ (2.3) ^disclosed $____ (2.3) ^undisclosed $____ (2.3) ^endorsed $____ (2.3) ^study$ ____ (2.2) ^Study$ ____ (2.2) ^diagnosed $____ (2.2) ^captured $____ (2.2) ^featured $____ (2.2) ^tortured $____ (2.2) ^manufactured $____ (2.2) ^fractured $____ (2.2) ^structured $____ (2.2) ^pictured $____ (2.2) ^excuse$ ____ (2.2) ^accuse$ ____ (2.2) ^Syracuse$ ____ (2.2) ^custody$ ____ (2.1) ^Austi n$____ (2.1) ^Ray$ ____(2.1) Filter 4 (bias = -0.37) #
F
w
7
z
S
o
4
-
6
U
8
m
Q
u
C
T
5
\195
j
O
h
v
V
K
\194
G
X
r
x
J
H
E
9
a
,
p
L
i
^
b
R
p
&
f
Y
y
U
e
z
i
L
w
u
x
J
t
D
m
9
W
\194
h
Q
F
l
g
\195
n
Z
\$
0
k
7
q
(
?
)
\163
/
c
T
f
Q
x
H
o
"
L
q
s
k
l
E
n
X
K
\163
m
I
,
r
5
t
S
W
z
g
/
v
a
Y
A
D
p
e
y
$
J
u
i
w
D
K
c
W
j
m
d
U
C
a
0
f
v
y
\194
B
g
b
l
i
7
H
t
Z
h
P
F
k
.
O
S
\195
5
r
T
E
R
X
x
Q
i
X
-
\194
u
Y
k
8
y
7
f
L
g
9
w
"
p
/
m
6
H
l
s
$
n
5
t
0
e
K
h
N
r
D
U
P
d
Non-zero for 16.7% of words.
^Real $____(3.0) ^Real ly$____(3.0) ^FRANC ISCO$____ (2.9) ^injury$ ____ (2.8) ^jury$ ____ (2.8) ^Form$ ____ (2.7) ^Farm$ ____ (2.6) ^luxury$ ____ (2.5) ^Refo rm$____(2.4) ^Ray$ ____(2.3) ^Cuba$ ____ (2.3) ^Dem$ ____(2.3) ^400m$ ____ (2.3) ^Ashley$ ____ (2.3) ^500m$ ____ (2.2) ^Club$ ____ (2.2) ^Rep. $____(2.2) ^Rail $____(2.2) ^flew$ ____ (2.1) ^YOU$ ____(2.1) ^Reme mber$____(2.1) ^UAW$ ____(2.1) ^Surel y$____ (2.1) ^Sure$ ____ (2.1) ^TEHRAN$ ____ (2.1) ^Law$ ____(2.1) ^procedural $____ (2.1) ^Deal $____(2.1) ^U.K. $____(2.1) ^util ity$____(2.1) ^util ities$____(2.1) ^CCTV$ ____ (2.1) ^fury$ ____ (2.0) ^Slam$ ____ (2.0) ^MOSCOW$ ____ (2.0) ^Read ing$____(2.0) ^Read $____(2.0) ^duty$ ____ (2.0) ^ATP$ ____(2.0) ^Hubbl e$____ (2.0) Filter 5 (bias = -0.49) #
k
-
C
f
Q
m
R
y
I
o
V
L
<BOS>
w
P
e
Y
.
T
M
7
u
\194
E
X
s
9
x
B
h
"
N
r
K
p
J
z
l
0
d
K
g
8
-
f
u
6
t
X
E
B
w
P
.
F
T
,
j
9
r
Q
h
/
v
7
z
Z
s
5
o
L
c
x
?
V
G
3
A
;
i
U
p
Y
0
Q
e
u
w
"
j
'
5
R
J
H
2
m
o
k
c
$
g
b
v
/
n
\194
D
E
1
d
i
x
G
U
w
y
p
u
f
H
-
D
j
L
k
8
o
d
t
Q
i
"
l
1
J
h
g
Z
n
7
v
6
S
a
'
4
z
/
m
b
x
-
F
o
B
R
A
J
f
g
L
G
t
O
k
Y
a
u
h
d
x
\195
U
$
y
I
m
j
K
r
,
'
c
1
b
i
C
z
N
"
e
Non-zero for 10.4% of words.
^Blu- ray$____(4.1) ^Secretary- General$____ (3.8) ^19th- century$____ (3.5) ^cash- strapped$____ (3.4) ^play- off$____ (3.4) ^day- to-day$____(3.2) ^record- breaking$____ (3.2) ^Budg et$____(3.0) ^hard- line$____ (2.9) ^pre- tax$____(2.9) ^sub- prime$____(2.8) ^mid- 1990s$____(2.7) ^prime- time$____ (2.7) ^Coca- Cola$____ (2.6) ^Buy$ ____(2.5) ^Budd hist$____(2.5) ^KABUL$ ____ (2.4) ^third- quarter$____ (2.4) ^third- party$____ (2.4) ^third- largest$____ (2.4) ^Wal- Mart$____(2.4) ^part- time$____ (2.4) ^run- up$____(2.4) ^run- off$____(2.4) ^old- fashioned$____(2.4) ^pro- democracy$____(2.4) ^forward- looking$____ (2.3) ^award- winning$____ (2.3) ^cross- border$____ (2.2) ^buyo ut$____(2.2) ^fixed- rate$____ (2.2) ^health- care$____ (2.1) ^Port- au-Prince$____ (2.1) ^start- up$____ (2.0) ^same- sex$____ (2.0) ^same- store$____ (2.0) ^man- made$____(2.0) ^working- class$____ (2.0) ^Judg e$____(1.9) ^York- based$____ (1.9) Filter 6 (bias = -0.51) #
<BOS>
g
Q
h
\194
i
"
H
X
r
W
u
K
-
8
j
9
m
6
A
/
n
^
p
v
y
N
k
J
w
l
\195
1
b
m
E
r
s
/
v
P
x
;
e
C
-
Q
w
X
4
K
2
Y
u
y
g
H
5
l
)
'
0
!
d
n
j
p
S
A
h
M
3
k
\$
Q
o
2
l
W
h
a
r
E
m
X
i
I
j
8
Y
Z
p
6
g
"
J
.
-
U
M
d
f
9
R
w
T
\194
k
\163
O
4
y
7
S
a
n
U
g
z
j
E
M
T
'
L
-
d
V
B
C
O
Z
K
r
D
k
A
h
l
H
W
F
v
R
I
m
t
c
,
$
\163
4
2
.
W
r
X
k
\194
f
6
b
5
P
$
s
Q
U
8
l
"
u
Z
p
/
R
4
\195
M
F
2
a
7
m
1
-
9
A
j
E
J
Non-zero for 22.1% of words.
^mad$ ____(3.0) ^CIT$ ____(2.8) ^AZUZ $____(2.8) ^reac h$____(2.7) ^reac hed$____(2.7) ^reac tion$____(2.7) ^reac hing$____(2.7) ^Brad$ ____ (2.7) ^Peac e$____(2.6) ^CEO$ ____(2.6) ^rat$ ____(2.5) ^Cal$ ____(2.5) ^Pat$ ____(2.4) ^CIA$ ____(2.4) ^Andrea$ ____ (2.4) ^Tea$ ____(2.4) ^red$ ____(2.4) ^CITY $____(2.4) ^Had$ ____(2.4) ^area$ ____ (2.3) ^mean s$____(2.3) ^mean $____(2.3) ^mean t$____(2.3) ^mean ing$____(2.3) ^Juarez$ ____ (2.3) ^Gomez$ ____ (2.3) ^pad$ ____(2.3) ^Mad$ ____(2.3) ^crazy $____ (2.3) ^Korea$ ____ (2.3) ^matc h$____(2.2) ^matc hes$____(2.2) ^matc hed$____(2.2) ^matc hing$____(2.2) ^lazy $____(2.2) ^tea$ ____(2.2) ^map$ ____(2.1) ^peac e$____(2.1) ^peac eful$____(2.1) ^peac ekeepers$____(2.1) Filter 7 (bias = -0.48) #
t
9
k
8
s
0
m
J
f
6
'
7
S
5
A
1
i
3
.
Z
U
D
g
X
a
K
p
2
r
L
F
N
u
G
I
o
w
4
-
<EOS>
?
p
.
T
!
P
(
v
Z
0
u
i
Q
l
N
J
:
D
s
j
w
B
r
;
'
6
&
d
\$
t
y
1
U
5
-
z
A
k
/
K
A
f
I
m
a
F
z
M
U
v
1
'
2
x
\195
j
3
-
Y
c
O
.
R
e
K
y
H
h
L
g
J
k
B
d
,
b
7
t
l
S
Q
y
\194
m
S
c
$
p
"
w
7
a
Y
f
R
U
I
g
X
L
h
u
n
b
A
_
T
B
e
v
Z
p
M
d
G
v
Y
q
K
t
S
l
V
-
U
I
/
P
Q
a
3
T
$
f
4
x
L
i
"
k
s
e
8
j
y
\163
D
n
Non-zero for 15.7% of words.
^extra$_ ___ (2.9) ^orchestra$_ ___ (2.9) ^Orchestra$_ ___ (2.9) ^4.1$_ ___ (2.9) ^NASA $____(2.9) ^4.2$_ ___ (2.8) ^NASC AR$____(2.8) ^PARIS$ ____ (2.8) ^2.1$_ ___ (2.8) ^2.2$_ ___ (2.7) ^actual$ ____ (2.7) ^virtual$ ____ (2.7) ^intellectual$ ____ (2.7) ^mutual$ ____ (2.7) ^3.1$_ ___ (2.7) ^1.1$_ ___ (2.7) ^5.1$_ ___ (2.7) ^1.25$ ____ (2.7) ^3.2$_ ___ (2.6) ^6.1$_ ___ (2.6) ^1.2$_ ___ (2.6) ^5.2$_ ___ (2.6) ^7.2$_ ___ (2.6) ^6.2$_ ___ (2.6) ^Massa$_ ___ (2.6) ^Melissa$_ ___ (2.6) ^Vanessa$_ ___ (2.6) ^Basra$_ ___ (2.5) ^4.3$_ ___ (2.5) ^0.1$_ ___ (2.5) ^0.2$_ ___ (2.5) ^AAA$_ ___ (2.5) ^central$ ____ (2.4) ^Central$ ____ (2.4) ^neutral$ ____ (2.4) ^2.3$_ ___ (2.4) ^Nicaragua$_ ___ (2.4) ^Antigua$_ ___ (2.4) ^FAA$_ ___ (2.3) ^usual$ ____ (2.3) Filter 8 (bias = -0.68) #
/
k
A
x
l
f
I
p
L
b
D
v
<BOS>
F
N
m
O
y
o
h
.
P
z
V
5
B
^
c
Q
'
<EOS>
e
\163
i
"
q
z
b
C
f
G
B
&
h
g
F
s
x
-
q
c
e
R
m
D
k
I
W
0
H
o
;
d
M
1
y
O
v
n
\$
U
N
5
X
2
a
x
k
\194
g
6
y
7
r
9
T
8
m
5
t
l
p
L
w
v
c
Q
O
4
M
X
U
d
i
"
G
0
P
J
A
.
E
/
K
Y
\163
a
j
A
S
z
f
L
F
B
r
U
e
Z
p
.
O
w
M
q
t
W
P
b
-
v
'
n
E
X
g
/
k
Y
y
\195
s
6
R
1
\163
v
i
\194
f
Q
y
"
H
c
p
.
A
X
P
0
,
9
s
q
L
$
m
T
F
z
l
8
r
D
S
W
h
'
n
Y
O
d
U
b
K
Non-zero for 22.0% of words.
^slav ery$____(4.6) ^slav e$____(4.6) ^slav es$____(4.6) ^Chav ez$____(4.4) ^enclav e$____ (4.3) ^glac iers$____(3.5) ^Cliv e$____(3.3) ^full-bac k$____ (3.2) ^Asda$ ____ (3.2) ^Conv ention$____(3.2) ^Coac h$____(3.0) ^Mousav i$____ (3.0) ^unav ailable$____(3.0) ^savv y$____(3.0) ^canv as$____(2.9) ^back-to-bac k$____ (2.9) ^U.S.-bac ked$____ (2.8) ^glov es$____(2.8) ^glov e$____(2.8) ^short-liv ed$____ (2.7) ^Endeav our$____ (2.7) ^salv age$____(2.7) ^conv icted$____(2.7) ^conv ention$____(2.7) ^conv ersation$____(2.7) ^conv inced$____(2.7) ^snac ks$____(2.7) ^snac k$____(2.7) ^oliv e$____(2.7) ^Oliv er$____(2.6) ^coac h$____(2.6) ^coac hes$____(2.6) ^coac hing$____(2.6) ^coac hed$____(2.6) ^Plac e$____(2.6) ^glad $____(2.6) ^solv e$____(2.5) ^solv ed$____(2.5) ^solv ing$____(2.5) ^Doha$ ____ (2.5) Filter 9 (bias = -0.35) #
f
Y
F
q
s
1
<BOS>
h
S
H
e
T
j
g
E
i
K
u
L
p
M
a
5
W
N
v
,
k
l
c
t
y
Q
0
P
n
.
\195
m
o
y
v
H
b
d
g
/
J
D
j
,
w
!
k
Q
E
(
V
1
z
?
x
:
-
L
G
C
)
N
s
P
B
U
S
\$
0
&
e
8
i
-
c
J
A
w
D
v
t
b
F
V
S
W
y
9
C
X
h
Z
d
i
L
B
g
Y
.
M
r
\195
O
6
s
k
l
m
,
u
T
q
\163
"
d
Q
g
X
l
M
.
W
-
V
A
\194
n
Y
a
B
u
9
p
$
L
K
z
S
s
8
D
6
y
k
o
4
c
h
_
w
Q
g
X
j
W
E
/
b
Z
s
n
v
N
e
$
k
\194
l
8
T
9
p
6
d
K
f
3
h
7
u
5
F
2
m
t
r
x
Non-zero for 10.8% of words.
^best-kn own$____ (3.3) ^FOX$_ ___ (2.2) ^1-3$ ____(2.2) ^symbo l$____ (2.2) ^symbo lic$____ (2.2) ^symbo ls$____ (2.2) ^save$ ____ (2.2) ^saw$_ ___ (2.2) ^oversaw$_ ___ (2.2) ^Warsaw$_ ___ (2.2) ^Save$ ____ (2.2) ^SUV$_ ___ (2.1) ^Kyi$_ ___ (2.1) ^frien ds$____ (2.0) ^frien d$____ (2.0) ^frien dly$____ (2.0) ^girlfrien d$____ (2.0) ^fake$ ____ (2.0) ^Frien ds$____ (2.0) ^fame$ ____ (2.0) ^ingredien ts$____ (2.0) ^ingredien t$____ (2.0) ^Fame$ ____ (2.0) ^7-6$ ____(2.0) ^five$ ____ (1.9) ^heaven $____ (1.9) ^Five$ ____ (1.9) ^U.S.-ba cked$____ (1.9) ^leave$ ____ (1.9) ^sub$_ ___ (1.9) ^sank$ ____ (1.8) ^jaw$_ ___ (1.8) ^1-2$ ____(1.8) ^few$_ ___ (1.8) ^curfew$_ ___ (1.8) ^3-6$ ____(1.8) ^Few$_ ___ (1.8) ^Punjab$_ ___ (1.8) ^well-kn own$____ (1.8) ^6-4$ ____(1.8) Filter 10 (bias = -0.50) #
X
-
<BOS>
f
Y
u
Q
s
7
y
l
c
6
w
\194
n
V
r
L
o
5
'
^
k
/
g
A
d
B
.
W
v
4
m
0
x
8
p
J
t
D
k
.
U
e
i
d
Y
j
V
t
b
\$
B
-
R
(
;
F
m
N
%
c
s
E
a
\194
)
X
P
l
H
q
h
\163
G
?
u
I
\195
Q
p
X
g
Z
h
V
y
B
i
\194
o
b
j
/
-
W
r
K
t
z
d
$
T
"
c
.
u
9
O
U
D
6
e
L
f
8
1
H
I
.
T
m
H
s
r
z
P
-
t
x
k
w
p
L
F
c
Q
v
q
U
j
G
7
Z
1
o
X
u
O
b
e
g
i
'
,
n
R
a
O
q
S
n
M
d
K
A
o
a
m
1
s
g
f
D
"
C
Y
h
W
H
'
I
$
0
U
c
G
7
E
p
-
v
/
.
2
F
Non-zero for 21.2% of words.
^LeBro n$____ (3.4) ^Years $____ (3.3) ^Year$ ____ (3.0) ^Darf ur$____(2.9) ^DVDs $____(2.8) ^DUP$ ____(2.8) ^leaks $____ (2.8) ^celebri ty$____ (2.8) ^celebri ties$____ (2.8) ^Celebri ty$____ (2.8) ^clear$ ____ (2.7) ^nuclear$ ____ (2.7) ^unclear$ ____ (2.7) ^Nuclear$ ____ (2.7) ^Dako ta$____(2.7) ^earm arked$____(2.7) ^earm arks$____(2.7) ^ears $____(2.6) ^nuclear- armed$____ (2.6) ^Bears $____ (2.6) ^DVD$ ____(2.5) ^leak$ ____ (2.5) ^bleak$ ____ (2.5) ^Daim ler$____(2.4) ^leap$ ____ (2.4) ^ear$ ____(2.4) ^FBI$ ____(2.3) ^Darw in$____(2.3) ^Yeah$ ____ (2.3) ^cleari ng$____ (2.3) ^Bear$ ____ (2.3) ^milesto ne$____ (2.2) ^Charlesto n$____ (2.2) ^Sears $____ (2.2) ^NATO $____(2.2) ^eat$ ____(2.1) ^cleare d$____ (2.1) ^cleare r$____ (2.1) ^dollars $____ (2.1) ^Lewis $____ (2.1) Filter 11 (bias = -0.54) #
Q
v
X
x
Z
g
/
h
P
k
M
b
6
c
8
t
3
o
K
.
^
p
"
q
2
a
H
w
<BOS>
l
U
-
7
A
L
B
4
z
1
r
U
x
u
F
m
S
H
p
T
f
Y
\$
;
e
M
j
-
h
R
c
Z
5
&
g
z
0
/
N
)
q
\195
l
P
.
i
n
!
b
"
7
B
-
V
g
n
G
k
E
\194
d
6
r
,
.
9
O
X
s
W
p
C
y
H
\163
K
z
5
e
M
o
4
c
F
u
/
j
N
D
7
w
.
i
-
k
Q
p
b
T
l
C
L
1
N
t
'
B
"
h
/
y
r
c
\194
4
u
H
$
F
\195
P
R
0
,
3
5
2
-
A
'
U
9
t
o
T
"
F
n
L
J
D
W
B
$
c
\194
z
x
_
3
a
R
E
7
y
C
k
l
S
m
K
Non-zero for 10.8% of words.
^UBS$ ____(4.5) ^HBO$ ____(3.7) ^Unli ke$____(3.6) ^undo ubtedly$____(3.6) ^unan imously$____(3.5) ^unan imous$____(3.5) ^unan swered$____(3.5) ^unli kely$____(3.5) ^unli ke$____(3.5) ^unli mited$____(3.5) ^mid- 1990s$____(3.5) ^fund- raising$____ (3.4) ^RBI$ ____(3.3) ^unaw are$____(3.3) ^unav ailable$____(3.2) ^unau thorized$____(3.2) ^undi sclosed$____(3.1) ^Unle ss$____(3.1) ^unbe aten$____(3.1) ^unbe lievable$____(3.1) ^unsu ccessful$____(3.1) ^unsu re$____(3.1) ^unsu ccessfully$____(3.1) ^unsu stainable$____(3.1) ^Unfo rtunately$____(3.1) ^unle ss$____(3.0) ^unle ashed$____(3.0) ^unex pected$____(3.0) ^unex pectedly$____(3.0) ^'Bri en$____(3.0) ^RBS$ ____(3.0) ^unfo rtunate$____(3.0) ^unfo rtunately$____(3.0) ^unre st$____(3.0) ^unre lated$____(3.0) ^unre alistic$____(3.0) ^unof ficial$____(2.9) ^TB$_ ___(2.9) ^Quinn $____ (2.9) ^unwi lling$____(2.9) Filter 12 (bias = -0.51) #
<BOS>
y
s
L
\194
p
"
n
Q
m
S
D
R
A
'
r
W
K
E
f
Y
l
^
h
I
P
v
c
k
Z
g
H
e
q
\195
N
v
L
k
K
g
y
p
O
T
(
j
/
-
\$
b
,
V
U
i
f
)
3
0
8
q
o
C
%
t
&
z
S
c
Q
d
:
'
E
l
7
o
4
y
2
O
I
m
V
f
6
K
Q
M
q
T
X
c
9
p
Z
t
a
r
8
G
H
S
5
D
b
l
1
-
W
z
\194
'
n
s
Y
v
H
c
A
d
V
w
l
-
r
.
S
x
Q
f
R
z
7
o
X
u
L
D
h
e
j
t
i
\163
3
T
4
y
/
E
P
a
0
V
D
v
t
k
y
b
r
Y
A
W
O
9
d
Z
L
B
l
'
c
J
e
\194
F
x
.
-
o
6
N
i
p
"
E
w
T
X
S
4
\163
Non-zero for 13.0% of words.
^NHS$ ____(3.2) ^Salv ador$____(3.1) ^NBA$ ____(3.1) ^Kirk $____(3.0) ^Kirk uk$____(3.0) ^NHL$ ____(3.0) ^Sark ozy$____(2.9) ^salv age$____(2.9) ^Falk irk$____(2.8) ^Lank a$____(2.8) ^Lank an$____(2.8) ^soari ng$____ (2.7) ^MSNBC$ ____ (2.7) ^LAS$ ____(2.7) ^Lamb ert$____(2.7) ^NFL$ ____(2.7) ^Nikk ei$____(2.6) ^soar$ ____ (2.6) ^Walk er$____(2.6) ^Walk $____(2.6) ^ANGELES$ ____ (2.6) ^BERLIN$ ____ (2.5) ^Univ ersity$____(2.5) ^Univ ersal$____(2.5) ^Latv ia$____(2.5) ^necessari ly$____ (2.5) ^Mark $____(2.4) ^Mark et$____(2.4) ^Mark ets$____(2.4) ^Mark s$____(2.4) ^Mark eting$____(2.4) ^Karz ai$____(2.4) ^Onli ne$____(2.4) ^Serv ice$____(2.4) ^Serv ices$____(2.4) ^dismissal$ ____ (2.4) ^NFC$ ____(2.3) ^Silv er$____(2.3) ^Silv a$____(2.3) ^Silv io$____(2.3) Filter 13 (bias = -0.58) #
M
x
T
a
<BOS>
f
Y
F
G
p
m
n
X
h
O
d
Z
A
"
,
\194
q
^
s
W
r
-
b
D
l
w
P
J
L
z
I
V
B
K
y
d
k
e
R
D
r
t
n
E
U
S
B
\$
Z
6
u
l
C
O
b
\194
Y
j
A
p
\195
X
V
F
)
f
?
W
H
\163
g
5
w
:
J
z
-
U
j
m
i
.
h
c
x
A
e
L
3
D
4
/
f
Z
p
T
E
C
J
K
S
G
7
B
2
Q
9
\194
1
t
F
l
H
M
5
8
p
N
T
Z
k
9
i
F
t
6
z
n
l
7
g
Q
m
4
O
5
G
3
Y
2
r
.
b
X
S
/
A
M
P
L
o
"
s
\194
-
\194
w
"
b
Q
g
D
J
$
E
d
A
T
r
/
n
c
e
C
\195
y
i
'
j
W
B
O
k
X
f
t
a
,
-
Y
s
8
_
Z
Non-zero for 21.5% of words.
^ATLANT A$____ (4.0) ^Tech$ ____ (3.7) ^6.8$ ____(3.7) ^SAN$ ____(3.5) ^5.8$ ____(3.5) ^2.8$ ____(3.4) ^6.6$ ____(3.3) ^homemad e$____ (3.2) ^5.9$ ____(3.2) ^6.7$ ____(3.2) ^0.8$ ____(3.1) ^4.8$ ____(3.1) ^Tenn$ ____ (3.1) ^6.4$ ____(3.1) ^2.9$ ____(3.1) ^5.6$ ____(3.0) ^6.5$ ____(3.0) ^DAX$ ____(3.0) ^Hamdan$ ____ (3.0) ^Macy$ ____ (3.0) ^EDF$ ____(3.0) ^2.6$ ____(2.9) ^6.3$ ____(2.9) ^5.7$ ____(2.9) ^6.2$ ____(2.9) ^memo$ ____ (2.9) ^SEOUL$ ____ (2.9) ^1.8$ ____(2.9) ^5.4$ ____(2.8) ^2.7$ ____(2.8) ^5.5$ ____(2.8) ^mean$ ____ (2.8) ^Mets$ ____ (2.8) ^Time$ ____ (2.8) ^3.8$ ____(2.8) ^0.9$ ____(2.8) ^4.9$ ____(2.7) ^2.4$ ____(2.7) ^7.6$ ____(2.7) ^2.5$ ____(2.7) Filter 14 (bias = -0.47) #
X
p
7
y
h
f
\194
U
6
O
Y
P
V
r
4
c
<BOS>
o
5
K
x
t
L
k
H
z
l
s
.
T
b
m
q
,
Q
C
8
w
9
i
S
v
G
B
O
k
s
u
Q
H
"
n
&
q
E
T
\$
i
8
m
5
w
(
J
:
t
F
b
7
\195
!
a
'
)
h
M
1
k
-
A
d
V
D
B
o
Q
u
C
f
W
l
Z
j
b
P
Y
J
q
e
a
O
H
m
X
y
n
L
w
p
4
E
h
v
N
s
$
T
y
l
M
v
Q
J
c
i
"
a
K
x
r
-
O
z
f
0
m
I
'
b
N
d
/
g
F
2
$
1
Z
6
C
q
t
7
D
u
.
5
i
.
k
x
Y
d
V
N
m
L
C
v
H
e
p
E
P
a
T
D
t
c
M
8
O
q
'
u
$
F
S
9
,
0
I
\163
W
-
o
Non-zero for 19.1% of words.
^Skyp e$____(4.4) ^Sky$ ____(4.3) ^sky$ ____(3.7) ^sayi ng$____(3.6) ^Say$ ____(3.3) ^Sark ozy$____(3.3) ^Safi na$____(3.3) ^SAN$ ____(3.2) ^HSBC$ ____ (3.0) ^GAO$ ____(2.8) ^Gay$ ____(2.8) ^sack ed$____(2.8) ^sack $____(2.8) ^sack s$____(2.8) ^risky$ ____ (2.7) ^say$ ____(2.7) ^'Bri en$____(2.7) ^shri nking$____(2.6) ^shri nk$____(2.6) ^shri ne$____(2.6) ^SAP$ ____(2.6) ^Stri p$____(2.5) ^shy$ ____(2.5) ^hybri d$____ (2.5) ^hybri ds$____ (2.5) ^shaky$ ____ (2.5) ^Simi larly$____(2.5) ^Simi lar$____(2.5) ^payi ng$____(2.4) ^layi ng$____(2.4) ^Sam$ ____(2.4) ^subscri bers$____ (2.4) ^subscri ption$____ (2.4) ^Sotheby$ ____ (2.4) ^Shei kh$____(2.3) ^Shei la$____(2.3) ^birthday$ ____ (2.3) ^Anthony$ ____ (2.3) ^Symphony$ ____ (2.3) ^heari ng$____ (2.3) Filter 15 (bias = -0.51) #
Q
-
C
v
A
f
Z
p
/
x
U
o
V
d
^
e
5
q
X
m
Y
u
S
b
7
h
G
i
<BOS>
y
4
r
L
T
J
k
w
I
F
-
S
q
h
!
f
d
s
p
L
Q
M
X
m
?
%
P
x
;
j
2
y
\195
A
1
B
a
\$
z
5
W
4
w
U
v
K
:
l
Z
p
X
o
L
O
6
g
M
c
Q
x
H
y
B
i
/
S
F
G
.
r
\194
h
V
s
D
k
7
-
N
f
J
'
2
R
9
C
q
z
j
y
l
c
S
U
Y
k
J
T
-
p
g
C
E
P
V
t
7
f
b
d
5
D
G
n
L
m
4
v
X
,
Q
a
\194
B
.
u
s
K
y
R
c
J
x
u
g
-
p
I
.
\195
m
E
h
r
n
U
F
T
Z
P
C
O
W
Y
5
o
V
z
w
l
A
j
f
9
4
N
X
B
Non-zero for 18.9% of words.
^26-y ear-old$____(3.5) ^16-y ear-old$____(3.5) ^AIDS$ ____ (3.4) ^Welc ome$____(3.2) ^welc ome$____(3.2) ^welc omed$____(3.2) ^welc oming$____(3.2) ^rely $____(3.1) ^rely ing$____(3.1) ^27-y ear-old$____(3.1) ^ally $____(3.1) ^17-y ear-old$____(3.1) ^Sally $____ (3.0) ^22-y ear-old$____(3.0) ^12-y ear-old$____(3.0) ^29-y ear-old$____(3.0) ^19-y ear-old$____(2.9) ^28-y ear-old$____(2.9) ^24-y ear-old$____(2.9) ^18-y ear-old$____(2.9) ^14-y ear-old$____(2.9) ^Analy sts$____ (2.9) ^Analy sis$____ (2.9) ^25-y ear-old$____(2.8) ^15-y ear-old$____(2.8) ^desc ribed$____(2.8) ^desc ribe$____(2.8) ^desc ribes$____(2.8) ^desc ribing$____(2.8) ^Insp ector$____(2.8) ^Insp $____(2.8) ^IAEA $____(2.7) ^36-y ear-old$____(2.6) ^Walc ott$____(2.6) ^CITY$ ____ (2.6) ^Sadly $____ (2.5) ^palm $____(2.5) ^Cable $____ (2.5) ^July $____(2.5) ^webc ast$____(2.5) Filter 16 (bias = -0.72) #
<BOS>
c
m
g
X
w
Q
E
/
0
\194
x
M
A
l
p
-
k
L
h
"
C
Y
a
^
G
o
n
s
2
5
i
1
8
m
9
t
7
-
6
w
3
k
1
.
5
s
4
g
2
z
X
'
0
l
N
v
Q
f
Z
u
"
b
D
T
K
i
L
A
\$
U
(
%
K
g
,
.
o
b
B
-
U
d
O
m
R
j
N
e
3
h
9
q
5
\163
i
v
f
r
/
p
s
l
C
V
S
x
W
X
I
F
1
c
d
g
,
b
y
w
6
z
W
G
1
m
8
k
H
-
"
J
Q
.
7
r
\194
V
2
c
4
j
F
\195
3
A
D
Z
X
R
I
v
/
l
k
D
b
d
h
o
B
O
x
-
V
/
F
c
r
1
A
z
a
G
q
K
f
y
S
M
H
2
Y
0
g
5
R
6
i
L
p
\194
'
$
Non-zero for 17.4% of words.
^13th $____(4.0) ^Rodh am$____(3.9) ^19th $____(3.9) ^19th -century$____(3.9) ^15th $____(3.9) ^Noth ing$____(3.7) ^83$_ ___(3.6) ^11th $____(3.6) ^12th $____(3.6) ^Hopk ins$____(3.6) ^60th $____(3.6) ^89$_ ___(3.5) ^30th $____(3.5) ^18th $____(3.5) ^14th $____(3.5) ^85$_ ___(3.5) ^16th $____(3.5) ^25th $____(3.5) ^1964 $____(3.4) ^Rodr iguez$____(3.4) ^Cadb ury$____(3.4) ^1984 $____(3.3) ^Dodg ers$____(3.3) ^Dodg e$____(3.3) ^10th $____(3.3) ^81$_ ___(3.3) ^50th $____(3.3) ^82$_ ___(3.3) ^Diamondb acks$____ (3.3) ^1969 $____(3.2) ^80,0 00$____(3.2) ^40th $____(3.2) ^17th $____(3.2) ^75,0 00$____(3.2) ^49er s$____(3.2) ^Lodg e$____(3.2) ^Look $____(3.2) ^Look ing$____(3.2) ^1967 $____(3.2) ^1974 $____(3.2) Filter 17 (bias = -0.64) #
2
m
8
h
3
b
9
l
U
t
1
.
K
g
G
k
5
v
I
j
O
q
R
T
Q
x
C
f
z
e
Z
H
7
r
6
-
P
A
4
M
E
m
e
n
"
V
\$
Z
r
z
D
i
t
C
T
K
j
B
(
p
N
A
\163
k
d
l
u
U
O
b
-
%
S
w
Q
L
h
!
?
a
C
E
n
h
P
S
Z
j
z
e
I
g
/
o
U
x
p
F
X
s
Q
r
'
O
k
u
,
4
K
-
V
N
m
J
\194
Y
d
L
a
G
t
G
f
L
N
d
e
z
W
U
S
D
E
m
F
g
B
Z
w
P
q
C
Q
l
k
u
I
1
"
Y
O
\195
,
0
M
J
r
-
j
p
\194
f
Q
r
"
n
$
o
'
J
X
N
Y
y
.
B
W
p
V
K
i
w
\195
e
P
h
_
H
3
F
Non-zero for 31.6% of words.
^Kent$ ____ (4.6) ^Gene$ ____ (4.3) ^sent$ ____ (4.2) ^present$ ____ (4.2) ^represent$ ____ (4.2) ^consent$ ____ (4.2) ^Arsene$ ____ (3.9) ^ECB$ ____(3.8) ^President$ ____ (3.7) ^president$ ____ (3.7) ^incident$ ____ (3.7) ^independent$ ____ (3.7) ^student$ ____ (3.7) ^accident$ ____ (3.7) ^Genev a$____ (3.7) ^percent$ ____ (3.6) ^cent$ ____ (3.6) ^recent$ ____ (3.6) ^innocent$ ____ (3.6) ^SOURCE$ ____ (3.5) ^Gen.$ ____ (3.4) ^represents $____ (3.3) ^presents $____ (3.3) ^scene$ ____ (3.3) ^went$ ____ (3.3) ^underwent$ ____ (3.3) ^worsened $____ (3.2) ^Gone$ ____ (3.2) ^seat$ ____ (3.2) ^permanent$ ____ (3.2) ^prominent$ ____ (3.2) ^opponent$ ____ (3.2) ^continent$ ____ (3.2) ^component$ ____ (3.2) ^sect$ ____ (3.2) ^FRANCISCO$ ____ (3.1) ^ESPN$ ____ (3.0) ^zone$ ____ (3.0) ^eurozone$ ____ (3.0) ^Zone$ ____ (3.0) Filter 18 (bias = -0.33) #
S
k
s
T
N
m
o
p
3
b
5
P
<BOS>
v
4
g
8
c
O
B
"
z
9
t
W
V
-
C
E
D
Q
r
7
A
x
q
,
U
6
\195
X
t
V
s
b
o
Z
i
Q
O
M
,
Y
p
!
I
6
U
?
f
7
d
)
u
8
y
\194
a
L
S
"
z
9
-
m
%
.
C
:
c
t
-
S
b
O
x
A
v
T
u
D
J
,
\195
/
q
C
r
M
n
Q
R
$
9
K
a
F
f
5
p
W
d
k
P
h
o
p
B
g
v
I
u
G
U
O
x
P
N
r
M
d
L
i
\194
2
f
1
h
\163
R
y
o
-
9
X
.
Q
b
e
F
w
m
E
J
7
Y
H
c
U
x
W
f
Z
j
X
F
i
S
1
o
Y
r
2
'
m
.
a
t
w
v
L
s
6
C
u
g
3
p
K
-
/
e
T
h
A
n
Non-zero for 18.1% of words.
^bypa ss$____(3.8) ^bipa rtisan$____(3.5) ^obtai ned$____ (3.3) ^obtai n$____ (3.3) ^obtai ning$____ (3.3) ^begi n$____(3.0) ^begi nning$____(3.0) ^begi ns$____(3.0) ^BAGH DAD$____(2.8) ^Vega s$____(2.8) ^bega n$____(2.7) ^outpu t$____ (2.6) ^begu n$____(2.6) ^espi onage$____(2.5) ^conspi racy$____ (2.5) ^conspi ring$____ (2.5) ^mortga ge$____ (2.4) ^mortga ges$____ (2.4) ^Mortga ge$____ (2.4) ^mortga ge-backed$____ (2.4) ^shipm ents$____ (2.4) ^shipm ent$____ (2.4) ^Veri zon$____(2.4) ^VEGA S$____(2.3) ^Ossetia $____ (2.3) ^big$ ____(2.3) ^empi re$____(2.3) ^Vogu e$____(2.3) ^bogu s$____(2.3) ^problem s$____ (2.2) ^problem $____ (2.2) ^problem atic$____ (2.2) ^October$ ____ (2.2) ^sober$ ____ (2.2) ^Verm ont$____(2.2) ^NYSE$ ____ (2.1) ^contri buted$____ (2.1) ^contri butions$____ (2.1) ^contri bution$____ (2.1) ^contri bute$____ (2.1) Filter 19 (bias = -0.70) #
<BOS>
y
R
m
V
e
j
L
k
H
S
D
Y
d
C
f
s
h
'
a
\194
q
z
M
J
.
9
N
7
K
I
u
Q
T
l
W
<EOS>
\163
0
p
X
g
W
c
Q
s
N
G
I
h
2
u
q
k
6
o
:
C
B
m
H
p
/
y
(
z
Z
-
7
'
K
v
9
S
5
x
3
d
4
R
O
k
K
C
E
n
X
u
G
F
J
h
o
A
W
c
e
U
2
'
3
s
p
t
6
R
-
.
5
B
l
H
0
v
w
a
L
x
"
b
T
g
M
A
D
n
v
a
"
s
N
i
K
I
o
w
e
C
f
z
\194
-
B
V
O
Z
t
b
W
l
F
p
c
U
8
k
E
H
S
.
h
w
x
k
L
I
o
t
S
-
8
r
5
b
y
m
Y
P
6
z
7
T
4
q
3
v
G
\195
1
e
0
n
9
U
D
B
\194
'
,
V
Non-zero for 17.1% of words.
^Noth ing$____(4.5) ^Neth erlands$____(4.3) ^Boeh ner$____(4.2) ^NOT$ ____(4.1) ^WASHINGTO N$____ (4.0) ^Both $____(4.0) ^26th $____(3.9) ^Beth $____(3.8) ^Beth esda$____(3.8) ^25th $____(3.8) ^20th $____(3.7) ^Saleh $____ (3.7) ^60th $____(3.6) ^With $____(3.5) ^With out$____(3.5) ^With in$____(3.5) ^27th $____(3.4) ^Xbox $____(3.4) ^Befo re$____(3.3) ^24th $____(3.3) ^noth ing$____(3.3) ^ANGEL ES$____ (3.2) ^Nich olas$____(3.2) ^Nich olson$____(3.2) ^Nich ols$____(3.2) ^Loth ian$____(3.2) ^like-fo r-like$____ (3.2) ^50th $____(3.1) ^30th $____(3.1) ^40th $____(3.1) ^Wood s$____(3.0) ^Wood $____(3.0) ^Wood ward$____(3.0) ^Wood y$____(3.0) ^12th $____(3.0) ^Moth er$____(3.0) ^Noel $____(3.0) ^Bloo mberg$____(3.0) ^Bloo d$____(3.0) ^Beyo nd$____(2.9) Filter 20 (bias = -0.50) #
P
h
d
x
D
B
U
o
Q
S
-
f
I
W
<BOS>
w
Z
N
z
A
R
i
G
t
C
q
X
a
u
v
\195
5
r
4
^
n
/
,
"
k
(
k
.
P
&
p
?
-
O
v
y
f
L
i
A
J
W
n
N
j
S
b
G
;
Q
I
E
V
"
)
c
R
/
\195
\$
d
o
'
!
m
i
U
H
.
W
s
X
b
V
f
I
u
1
L
Y
x
4
a
n
r
5
F
j
E
t
d
C
z
g
y
$
N
2
c
\194
R
6
\195
7
\163
B
p
W
g
t
G
q
j
N
P
a
-
Q
x
T
J
U
S
w
l
.
i
\194
h
I
d
v
f
A
y
"
0
u
r
/
F
$
s
H
o
Y
c
l
y
R
t
J
v
r
d
Q
m
L
w
7
s
X
k
\195
f
N
u
9
T
b
U
o
F
/
e
K
i
V
p
3
M
8
C
O
\163
Non-zero for 22.0% of words.
^crucial $____ (3.6) ^commercial $____ (3.4) ^commercial s$____ (3.4) ^Commercial $____ (3.4) ^commercial ly$____ (3.4) ^Giul iani$____(3.2) ^Donal d$____ (3.2) ^McDonal d$____ (3.2) ^Popul ar$____ (3.0) ^Lith uania$____(2.8) ^independentl y$____ (2.8) ^Anal ysts$____(2.8) ^Anal ysis$____(2.8) ^Ronal do$____ (2.8) ^Ronal d$____ (2.8) ^frontl ine$____ (2.8) ^All-Star $____ (2.8) ^With $____(2.8) ^With out$____(2.8) ^With in$____(2.8) ^Detroit$ ____ (2.8) ^Hitl er$____(2.8) ^D.C.$ ____ (2.7) ^Stal in$____(2.6) ^titl e$____(2.6) ^titl es$____(2.6) ^titl ed$____(2.6) ^Eith er$____(2.6) ^Witn esses$____(2.6) ^hit$ ____(2.6) ^controversial $____ (2.6) ^Anto nio$____(2.6) ^adopt$ ____ (2.6) ^Contr ol$____ (2.6) ^special $____ (2.5) ^especial ly$____ (2.5) ^special ist$____ (2.5) ^Special $____ (2.5) ^Dmitr y$____ (2.5) ^Garcia$ ____ (2.5) Filter 21 (bias = -0.55) #
W
-
h
P
X
s
H
R
q
d
A
f
B
o
4
p
w
'
V
r
e
U
b
C
Y
z
g
u
E
O
M
\195
Z
,
T
n
6
l
2
I
F
z
f
T
V
o
M
D
e
c
X
O
Z
G
\$
d
4
U
6
A
j
l
x
p
b
R
Q
g
:
u
W
t
'
r
n
0
m
a
5
Y
a
j
B
-
W
f
X
M
Q
o
Y
y
z
g
b
e
V
r
9
F
\194
t
q
s
6
m
U
c
A
S
7
u
2
D
8
O
Z
n
"
'
6
r
4
y
J
p
V
c
v
O
5
D
\194
o
9
g
i
P
B
.
W
\163
2
t
7
G
3
d
u
T
Y
f
s
C
w
N
Z
A
H
K
V
t
G
H
x
u
g
T
s
o
b
N
S
r
5
y
j
q
F
A
Z
B
6
D
7
h
8
,
z
U
4
n
X
I
E
R
p
\195
Q
O
Non-zero for 18.7% of words.
^Vaux hall$____(3.0) ^heave n$____ (3.0) ^Vaug han$____(2.9) ^Fans $____(2.7) ^fals e$____(2.7) ^fals ely$____(2.7) ^Mass achusetts$____(2.7) ^Mass $____(2.7) ^Mass a$____(2.7) ^Renais sance$____ (2.6) ^fans $____(2.6) ^heads $____ (2.5) ^warheads $____ (2.5) ^beans $____ (2.5) ^Mave ricks$____(2.3) ^fasc inating$____(2.3) ^reveals $____ (2.2) ^bass $____(2.2) ^Limbaug h$____ (2.2) ^Netflix $____ (2.2) ^Flig ht$____(2.1) ^wais t$____(2.1) ^heavi ly$____ (2.1) ^heavi er$____ (2.1) ^steals $____ (2.1) ^Weiss $____ (2.1) ^upheava l$____ (2.1) ^Webbe r$____ (2.1) ^flig ht$____(2.1) ^flig hts$____(2.1) ^heal$ ____ (2.0) ^signals $____ (2.0) ^Falc ons$____(2.0) ^defaul t$____ (2.0) ^defaul ts$____ (2.0) ^Abbas $____ (2.0) ^ambass ador$____ (2.0) ^embass y$____ (2.0) ^Embass y$____ (2.0) ^Ambass ador$____ (2.0) Filter 22 (bias = -0.45) #
L
-
K
v
Y
k
S
n
A
p
X
c
Q
t
/
j
O
d
W
u
G
g
U
'
8
C
5
q
6
r
Z
I
3
w
"
T
y
R
4
i
f
D
x
T
-
A
'
c
b
U
V
u
:
g
9
d
;
y
n
t
p
H
K
L
J
h
W
G
w
z
X
1
Q
C
N
)
B
0
6
E
5
m
4
p
7
v
3
z
6
k
F
-
8
.
9
b
2
a
j
c
Z
T
J
'
X
t
M
y
N
g
H
r
1
d
L
o
V
x
S
u
b
y
q
O
Y
o
k
f
h
D
V
s
Q
K
x
M
B
m
v
w
7
t
X
c
9
-
"
G
a
U
\194
d
R
z
H
,
W
e
r
L
c
I
y
j
o
l
v
P
m
H
"
J
M
r
G
F
W
e
'
7
K
E
z
A
Y
\195
\194
d
x
2
.
q
$
i
U
-
O
L
C
1
Non-zero for 17.4% of words.
^webc ast$____(3.6) ^Lisbo n$____ (3.0) ^lifebo at$____ (2.8) ^beac h$____(2.7) ^beac hes$____(2.7) ^fenc e$____(2.6) ^fenc es$____(2.6) ^peac e$____(2.5) ^peac eful$____(2.5) ^peac ekeepers$____(2.5) ^peac ekeeping$____(2.5) ^peac efully$____(2.5) ^frac tion$____(2.5) ^frac tured$____(2.5) ^frac ture$____(2.5) ^Beac h$____(2.4) ^onbo ard$____(2.4) ^hijac ked$____ (2.4) ^rebo unds$____(2.3) ^rebo und$____(2.3) ^rebo unded$____(2.3) ^LeBro n$____ (2.3) ^forc es$____(2.3) ^forc e$____(2.3) ^forc ed$____(2.3) ^forc ing$____(2.3) ^perc ent$____(2.3) ^perc entage$____(2.3) ^perc eived$____(2.3) ^perc eption$____(2.3) ^perc eptions$____(2.3) ^Kerc her$____(2.3) ^Peac e$____(2.3) ^impeac hment$____ (2.2) ^fluc tuations$____(2.2) ^exerc ise$____ (2.2) ^exerc ises$____ (2.2) ^exerc ising$____ (2.2) ^exerc ised$____ (2.2) ^Asia-Pac ific$____ (2.2) Filter 23 (bias = -0.45) #
g
x
H
f
Z
B
Y
p
-
v
<BOS>
a
j
k
M
c
X
o
^
b
Q
K
G
q
V
N
/
F
,
P
r
y
9
h
x
g
8
E
N
T
n
m
9
j
\194
k
5
r
(
%
6
P
W
b
,
U
o
i
7
z
/
-
\$
t
c
G
"
;
Q
l
C
e
?
J
C
-
D
w
d
v
Q
g
/
J
,
E
F
b
L
o
y
k
8
j
7
u
P
i
1
M
6
e
A
m
X
T
\194
Y
H
r
I
\195
5
R
x
D
f
r
S
H
W
d
K
I
B
g
v
C
b
u
s
1
m
T
o
A
w
R
V
n
5
P
6
c
\194
\195
"
t
,
y
9
q
'
Z
d
v
1
B
p
M
P
k
7
b
y
t
2
f
H
w
6
.
3
N
G
T
D
m
C
o
8
u
i
'
I
R
L
E
5
j
4
S
,
e
Non-zero for 22.0% of words.
^windfa ll$____ (2.9) ^McCoy $____ (2.9) ^NASD AQ$____(2.9) ^NASC AR$____(2.9) ^Nasd aq$____(2.7) ^dragonfl y.$____ (2.7) ^WASH INGTON$____(2.6) ^anxi ety$____(2.5) ^anxi ous$____(2.5) ^MySp ace$____(2.5) ^NASA $____(2.5) ^adop ted$____(2.5) ^adop t$____(2.5) ^adop tion$____(2.5) ^adop ting$____(2.5) ^Baghdad $____ (2.5) ^BCS$ ____(2.5) ^enjoyed $____ (2.5) ^wounded $____ (2.4) ^founded $____ (2.4) ^funded $____ (2.4) ^surrounded $____ (2.4) ^sounded $____ (2.4) ^rounded $____ (2.4) ^Navy $____(2.4) ^McCon nell$____ (2.4) ^navy $____(2.4) ^Indep endent$____ (2.3) ^Indep endence$____ (2.3) ^Wasp s$____(2.3) ^taxp ayers$____(2.2) ^taxp ayer$____(2.2) ^golf$ ____ (2.2) ^advi ce$____(2.2) ^advi ser$____(2.2) ^advi sers$____(2.2) ^advi sed$____(2.2) ^NHS$ ____(2.2) ^reminded $____ (2.2) ^navi gation$____(2.1) Filter 24 (bias = -0.46) #
W
j
K
u
O
k
X
g
"
F
2
t
Q
s
8
R
3
C
y
r
6
-
p
l
,
n
o
.
1
v
5
m
^
b
a
A
/
h
N
D
k
G
s
o
R
l
u
m
F
D
Q
L
I
c
U
g
"
K
H
z
\$
0
'
p
(
T
:
J
S
%
;
M
4
w
,
O
f
X
r
y
f
g
K
u
x
T
L
i
X
E
l
I
/
t
B
H
\194
k
m
1
F
w
Q
-
6
d
b
r
N
c
8
\163
M
U
V
h
5
A
9
2
w
x
.
p
Z
h
I
f
n
v
A
P
/
k
Q
y
N
S
X
T
2
d
$
i
H
o
F
b
Y
G
c
s
0
h
z
F
m
7
w
4
v
S
c
H
T
8
K
3
U
6
-
j
o
5
t
x
G
9
.
Q
M
1
k
N
p
L
g
,
'
2
\195
R
O
Non-zero for 17.3% of words.
^Oklah oma$____ (3.6) ^Walsh $____ (2.8) ^BERLIN $____ (2.8) ^YORK$_ ___ (2.7) ^Rowe $____(2.7) ^RBIs $____(2.5) ^Walte r$____ (2.5) ^Walte rs$____ (2.5) ^RBI$ ____(2.5) ^breakfas t$____ (2.4) ^FBI$ ____(2.4) ^alwa ys$____(2.4) ^nowh ere$____(2.3) ^Walt$ ____ (2.3) ^Off$_ ___ (2.3) ^Kumar $____ (2.3) ^SEOUL$_ ___ (2.2) ^Howe ver$____(2.2) ^offsh ore$____ (2.2) ^Iowa $____(2.2) ^Offer $____ (2.2) ^Welsh $____ (2.1) ^unwi lling$____(2.1) ^Warwi ckshire$____ (2.1) ^Waxma n$____ (2.1) ^Wayne $____ (2.1) ^Kuzne tsova$____ (2.1) ^If$_ ___(2.1) ^Afgh anistan$____(2.1) ^Afgh an$____(2.1) ^Afgh ans$____(2.1) ^Wales $____ (2.0) ^unwa nted$____(2.0) ^Tyson$ ____ (2.0) ^railwa y$____ (2.0) ^How$ ____(2.0) ^Wilts hire$____ (2.0) ^Thompson$ ____ (2.0) ^Simpson$ ____ (2.0) ^seasone d$____ (2.0) Filter 25 (bias = -0.50) #
m
h
<BOS>
r
z
x
M
q
t
n
T
H
K
a
w
N
E
R
U
1
v
7
s
8
j
u
G
y
V
9
O
o
X
.
e
3
D
d
S
A
K
d
B
g
f
1
M
D
N
h
o
C
;
c
W
p
O
!
S
y
J
u
w
G
X
)
"
a
/
H
Q
A
\194
?
,
\163
9
7
R
0
A
f
z
x
I
o
t
-
Q
M
w
J
g
F
.
h
T
y
a
u
c
j
X
P
W
m
q
3
C
9
E
8
\163
n
2
L
U
K
G
i
R
.
J
c
Y
g
9
y
P
t
Q
m
3
d
\195
e
7
h
V
v
I
w
"
A
r
x
$
a
K
D
X
F
8
\163
p
L
E
h
p
Y
z
H
O
u
t
4
P
9
I
6
c
x
r
\194
G
W
g
8
D
V
s
"
d
Z
l
7
C
M
K
q
E
b
T
B
w
X
A
Non-zero for 14.3% of words.
^left-h ander$____ (3.7) ^Stockh olm$____ (3.5) ^stockh olders$____ (3.5) ^BAGH DAD$____(3.4) ^BERL IN$____(3.1) ^McQu een$____(3.1) ^motiv ated$____ (3.0) ^automotiv e$____ (3.0) ^motiv e$____ (3.0) ^motiv ation$____ (3.0) ^motiv es$____ (3.0) ^Automotiv e$____ (3.0) ^McCh rystal$____(2.8) ^GMAC$ ____ (2.7) ^Kash mir$____(2.7) ^warh eads$____(2.6) ^Bash ir$____(2.6) ^WASH INGTON$____(2.6) ^Manh attan$____(2.6) ^Barb ara$____(2.6) ^Barb er$____(2.6) ^SAP$ ____(2.5) ^mock$ ____ (2.5) ^fash ion$____(2.5) ^fash ionable$____(2.5) ^Kabu l$____(2.5) ^N.J. $____(2.5) ^N.Y. $____(2.4) ^promotin g$____ (2.4) ^Lockh eed$____ (2.4) ^Emmanu el$____ (2.4) ^Nash ville$____(2.3) ^Nash $____(2.3) ^fabu lous$____(2.3) ^Bar$ ____(2.3) ^follow-u p$____ (2.3) ^demogra phic$____ (2.3) ^Netflix $____ (2.3) ^antitru st$____ (2.2) ^glamorou s$____ (2.2) Filter 26 (bias = -0.43) #
t
6
r
L
k
J
.
0
w
8
'
D
I
4
n
1
A
9
<BOS>
5
C
G
Q
3
f
7
a
Y
-
h
g
v
q
2
p
x
N
d
s
X
N
i
Z
h
Q
m
?
v
n
T
!
k
8
Y
(
S
r
t
9
p
/
-
X
g
I
l
K
x
.
s
2
u
&
%
D
j
\$
;
7
o
K
g
B
c
P
w
L
-
X
t
J
.
U
s
9
n
8
C
Y
j
Q
p
"
d
N
i
M
'
6
h
\195
A
b
v
R
I
/
S
f
z
X
c
Q
.
W
z
"
n
4
m
3
d
7
U
V
o
E
D
Y
s
S
u
9
C
H
-
6
l
8
A
2
t
e
v
F
w
j
L
$
a
m
1
l
c
Y
d
Q
p
/
n
S
x
V
v
.
h
$
0
z
y
L
q
A
8
X
9
b
2
M
D
C
3
7
4
F
Non-zero for 22.6% of words.
^AZUZ$ ____ (2.5) ^'Neil l$____ (2.5) ^ANGEL ES$____ (2.5) ^rebel s$____ (2.4) ^rebel $____ (2.4) ^rebel lion$____ (2.4) ^true$ ____ (2.4) ^stalem ate$____ (2.4) ^stabil ity$____ (2.3) ^profitabil ity$____ (2.3) ^accountabil ity$____ (2.3) ^instabil ity$____ (2.3) ^stabil ize$____ (2.3) ^Accountabil ity$____ (2.3) ^remem ber$____ (2.3) ^remem bered$____ (2.3) ^remem bers$____ (2.3) ^vulnerabil ity$____ (2.2) ^memorabil ia$____ (2.2) ^trail $____ (2.2) ^trail ed$____ (2.2) ^trail ing$____ (2.2) ^trail er$____ (2.2) ^trail s$____ (2.2) ^rarel y$____ (2.2) ^NHS$ ____(2.2) ^tree$ ____ (2.2) ^AIDS$ ____ (2.1) ^CBS$ ____(2.1) ^Rockefel ler$____ (2.1) ^extremel y$____ (2.1) ^CNN$_ ___ (2.1) ^knee$ ____ (2.0) ^NYSE $____(2.0) ^CNBC$ ____ (2.0) ^RBS$ ____(2.0) ^trees $____ (2.0) ^scrambl ing$____ (2.0) ^scrambl ed$____ (2.0) ^scrambl e$____ (2.0) Filter 27 (bias = -0.75) #
<BOS>
y
V
t
Z
p
X
f
Y
c
7
T
J
U
6
k
j
O
4
a
5
d
\194
P
-
,
9
\163
l
r
Q
s
^
o
<EOS>
K
L
E
H
F
Q
v
I
g
r
j
A
-
R
x
N
e
&
)
/
m
,
h
(
0
;
G
U
E
:
M
C
d
O
c
a
J
K
i
B
w
P
b
!
4
\194
f
Y
r
A
F
W
-
z
P
l
e
T
y
X
b
/
p
t
j
$
k
5
M
0
u
6
m
D
s
I
x
Q
E
'
R
J
D
f
G
x
d
B
c
m
1
F
O
k
z
b
r
V
R
S
P
v
0
M
C
h
y
w
o
t
\195
W
T
i
U
e
I
j
p
s
g
'
d
h
P
N
z
w
p
o
l
H
Q
r
D
M
\194
u
m
B
X
n
U
g
C
R
s
J
L
q
'
3
6
E
/
k
I
4
G
Y
$
i
Non-zero for 16.2% of words.
^Rodd ick$____(4.3) ^Card iff$____(3.5) ^Card inals$____(3.5) ^Card inal$____(3.5) ^Rudd $____(3.5) ^BEIJING$ ____ (3.4) ^Kidd $____(3.3) ^Rhod e$____(3.3) ^AIG$ ____(3.3) ^al-Qaed a$____ (3.1) ^Al-Qaed a$____ (3.1) ^Airp ort$____(3.1) ^Jazz$ ____ (3.1) ^rapp er$____(3.1) ^AIDS $____(3.1) ^al-Qaid a$____ (3.1) ^rada r$____(3.1) ^Conrad$ ____ (2.9) ^NYC$ ____(2.9) ^Atom ic$____(2.9) ^Hidd ink$____(2.9) ^todd ler$____(2.9) ^Qaed a$____(2.9) ^Girard i$____ (2.8) ^Airl ines$____(2.8) ^Hard $____(2.8) ^Hard y$____(2.8) ^ladd er$____(2.8) ^Aids $____(2.8) ^Nada l$____(2.8) ^Bird $____(2.8) ^railroad $____ (2.8) ^rid$ ____(2.8) ^hybrids $____ (2.8) ^rand om$____(2.8) ^rand omly$____(2.8) ^ripp ed$____(2.8) ^ATP$ ____(2.8) ^road $____(2.8) ^road s$____(2.8) Filter 28 (bias = -0.55) #
v
f
V
O
g
s
Y
,
T
K
Z
o
0
S
\194
N
X
a
D
r
7
E
j
y
M
p
H
U
c
w
C
t
6
F
u
A
1
I
q
x
f
b
y
g
,
Y
K
h
n
A
p
l
W
R
/
u
M
r
O
z
2
E
P
k
5
.
6
G
C
J
t
\195
1
V
d
L
3
v
\$
%
H
.
X
c
6
s
J
f
i
r
1
n
Y
o
T
'
W
t
4
C
2
x
P
N
3
A
V
S<