Source/JavaScriptCore/ChangeLog

 12016-07-20 Saam Barati <sbarati@apple.com>
 2
 3 op_add/ValueAdd should be an IC in all JIT tiers
 4 https://bugs.webkit.org/show_bug.cgi?id=159649
 5
 6 Reviewed by NOBODY (OOPS!).
 7
 8 This patch makes Add an IC inside all JIT tiers. It does so in a
 9 simple, but effective, way. We will try to generate an int+int add
 10 that will repatch itself if its type checks fail. Sometimes though,
 11 we have runtime type data saying that the add won't be int+int.
 12 In those cases, we will just generate a full snippet that doesn't patch itself.
 13 Other times, we may generate no inline code and defer to making a C call. A lot
 14 of this patch is just refactoring ResultProfile into what we're now calling ArithProfile.
 15 ArithProfile does everything ResultProfile used to do, and more. It records simple type
 16 data about the LHS/RHS operands it sees. This allows us to determine if an op_add
 17 has only seen int+int operands, etc. ArithProfile will also contain the ResultType
 18 for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits.
 19 This means instead of having a side table like we did for ResultProfile, we just
 20 inject the ArithProfile into the bytecode instruction stream. This makes asking
 21 for ArithProfile faster; we no longer need to lock around this operation.
 22
 23 The size of an Add has gone down on average, but we can still do better.
 24 We still generate a lot of code because we generate calls to the slow path.
 25 I think we can make this better by moving the slow path to a shared thunk
 26 system. This patch mostly lays the foundation for future improvements to Add,
 27 and a framework to move all other arithmetic operations to be typed-based ICs.
 28
 29 Here is some data I took on the average op_add/ValueAdd size on various benchmarks:
 30 | JetStream | Speedometer | Unity 3D |
 31 ------| -------------|-----------------------------
 32 Old | 189 bytes | 169 bytes | 192 bytes |
 33 ------| -------------|-----------------------------
 34 New | 148 bytes | 124 bytes | 143 bytes |
 35 ---------------------------------------------------
 36
 37 Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet
 38 generator as a member variable. To make a snippet an IC, you need to implement
 39 a generateInline(.) method, which generates the inline IC. Then, you need to
 40 generate the IC where you used to generate the snippet. When generating the
 41 IC, we need to inform JITMathIC of various data like we do with StructureStubInfo.
 42 We need to tell it about where the slow path starts, where the slow path call is, etc.
 43 When generating a JITMathIC, it may tell you that it didn't generate any code inline.
 44 This is a request to the user of JITMathIC to just generate a C call along the
 45 fast path. JITMathIC may also have the snippet tell it to just generate the full
 46 snippet instead of the int+int path along the fast path.
 47
 48 In subsequent patches, we can improve upon how we decide to generate int+int or
 49 the full snippet. I tried to get clever by having double+double, double+int, int+double,
 50 fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up
 51 generating a lot of code when I did this and ended up using more memory than just generating
 52 the full snippet. There is probably some way we can be clever and generate specialized fast
 53 paths that are more successful than what I tried implementing, but I think that's worth doing
 54 this in follow up patches once the JITMathIC foundation has landed.
 55
 56 This patch also fixes a bug inside the slow path lambdas in the DFG.
 57 Before, it was not legal to emit an exception check inside them. Now,
 58 it is. So it's now easy to define arbitrary late paths using the DFG
 59 slow path lambda API.
 60
 61 * CMakeLists.txt:
 62 * JavaScriptCore.xcodeproj/project.pbxproj:
 63 * bytecode/ArithProfile.cpp: Added.
 64 (JSC::ArithProfile::emitDetectNumericness):
 65 (JSC::ArithProfile::shouldEmitSetDouble):
 66 (JSC::ArithProfile::emitSetDouble):
 67 (JSC::ArithProfile::shouldEmitSetNonNumber):
 68 (JSC::ArithProfile::emitSetNonNumber):
 69 (WTF::printInternal):
 70 * bytecode/ArithProfile.h: Added.
 71 (JSC::ObservedType::ObservedType):
 72 (JSC::ObservedType::sawInt32):
 73 (JSC::ObservedType::isOnlyInt32):
 74 (JSC::ObservedType::sawNumber):
 75 (JSC::ObservedType::isOnlyNumber):
 76 (JSC::ObservedType::sawNonNumber):
 77 (JSC::ObservedType::isOnlyNonNumber):
 78 (JSC::ObservedType::isEmpty):
 79 (JSC::ObservedType::bits):
 80 (JSC::ObservedType::withInt32):
 81 (JSC::ObservedType::withNumber):
 82 (JSC::ObservedType::withNonNumber):
 83 (JSC::ObservedType::withoutNonNumber):
 84 (JSC::ObservedType::operator==):
 85 (JSC::ArithProfile::ArithProfile):
 86 (JSC::ArithProfile::fromInt):
 87 (JSC::ArithProfile::lhsResultType):
 88 (JSC::ArithProfile::rhsResultType):
 89 (JSC::ArithProfile::lhsObservedType):
 90 (JSC::ArithProfile::rhsObservedType):
 91 (JSC::ArithProfile::setLhsObservedType):
 92 (JSC::ArithProfile::setRhsObservedType):
 93 (JSC::ArithProfile::tookSpecialFastPath):
 94 (JSC::ArithProfile::didObserveNonInt32):
 95 (JSC::ArithProfile::didObserveDouble):
 96 (JSC::ArithProfile::didObserveNonNegZeroDouble):
 97 (JSC::ArithProfile::didObserveNegZeroDouble):
 98 (JSC::ArithProfile::didObserveNonNumber):
 99 (JSC::ArithProfile::didObserveInt32Overflow):
 100 (JSC::ArithProfile::didObserveInt52Overflow):
 101 (JSC::ArithProfile::setObservedNonNegZeroDouble):
 102 (JSC::ArithProfile::setObservedNegZeroDouble):
 103 (JSC::ArithProfile::setObservedNonNumber):
 104 (JSC::ArithProfile::setObservedInt32Overflow):
 105 (JSC::ArithProfile::setObservedInt52Overflow):
 106 (JSC::ArithProfile::addressOfBits):
 107 (JSC::ArithProfile::detectNumericness):
 108 (JSC::ArithProfile::lhsSawInt32):
 109 (JSC::ArithProfile::lhsSawNumber):
 110 (JSC::ArithProfile::lhsSawNonNumber):
 111 (JSC::ArithProfile::rhsSawInt32):
 112 (JSC::ArithProfile::rhsSawNumber):
 113 (JSC::ArithProfile::rhsSawNonNumber):
 114 (JSC::ArithProfile::observeLHSAndRHS):
 115 (JSC::ArithProfile::bits):
 116 (JSC::ArithProfile::hasBits):
 117 (JSC::ArithProfile::setBit):
 118 * bytecode/CodeBlock.cpp:
 119 (JSC::CodeBlock::dumpRareCaseProfile):
 120 (JSC::CodeBlock::dumpArithProfile):
 121 (JSC::CodeBlock::dumpBytecode):
 122 (JSC::CodeBlock::addStubInfo):
 123 (JSC::CodeBlock::addJITAddIC):
 124 (JSC::CodeBlock::findStubInfo):
 125 (JSC::CodeBlock::resetJITData):
 126 (JSC::CodeBlock::shrinkToFit):
 127 (JSC::CodeBlock::dumpValueProfiles):
 128 (JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset):
 129 (JSC::CodeBlock::arithProfileForBytecodeOffset):
 130 (JSC::CodeBlock::arithProfileForPC):
 131 (JSC::CodeBlock::couldTakeSpecialFastCase):
 132 (JSC::CodeBlock::dumpResultProfile): Deleted.
 133 (JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted.
 134 (JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted.
 135 (JSC::CodeBlock::ensureResultProfile): Deleted.
 136 * bytecode/CodeBlock.h:
 137 (JSC::CodeBlock::stubInfoBegin):
 138 (JSC::CodeBlock::stubInfoEnd):
 139 (JSC::CodeBlock::couldTakeSlowCase):
 140 (JSC::CodeBlock::numberOfResultProfiles): Deleted.
 141 * bytecode/MethodOfGettingAValueProfile.cpp:
 142 (JSC::MethodOfGettingAValueProfile::emitReportValue):
 143 * bytecode/MethodOfGettingAValueProfile.h:
 144 (JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile):
 145 * bytecode/ValueProfile.cpp:
 146 (JSC::ResultProfile::emitDetectNumericness): Deleted.
 147 (JSC::ResultProfile::emitSetDouble): Deleted.
 148 (JSC::ResultProfile::emitSetNonNumber): Deleted.
 149 (WTF::printInternal): Deleted.
 150 * bytecode/ValueProfile.h:
 151 (JSC::getRareCaseProfileBytecodeOffset):
 152 (JSC::ResultProfile::ResultProfile): Deleted.
 153 (JSC::ResultProfile::bytecodeOffset): Deleted.
 154 (JSC::ResultProfile::specialFastPathCount): Deleted.
 155 (JSC::ResultProfile::didObserveNonInt32): Deleted.
 156 (JSC::ResultProfile::didObserveDouble): Deleted.
 157 (JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted.
 158 (JSC::ResultProfile::didObserveNegZeroDouble): Deleted.
 159 (JSC::ResultProfile::didObserveNonNumber): Deleted.
 160 (JSC::ResultProfile::didObserveInt32Overflow): Deleted.
 161 (JSC::ResultProfile::didObserveInt52Overflow): Deleted.
 162 (JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted.
 163 (JSC::ResultProfile::setObservedNegZeroDouble): Deleted.
 164 (JSC::ResultProfile::setObservedNonNumber): Deleted.
 165 (JSC::ResultProfile::setObservedInt32Overflow): Deleted.
 166 (JSC::ResultProfile::setObservedInt52Overflow): Deleted.
 167 (JSC::ResultProfile::addressOfFlags): Deleted.
 168 (JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted.
 169 (JSC::ResultProfile::detectNumericness): Deleted.
 170 (JSC::ResultProfile::hasBits): Deleted.
 171 (JSC::ResultProfile::setBit): Deleted.
 172 (JSC::getResultProfileBytecodeOffset): Deleted.
 173 * bytecompiler/BytecodeGenerator.cpp:
 174 (JSC::BytecodeGenerator::emitBinaryOp):
 175 * dfg/DFGByteCodeParser.cpp:
 176 (JSC::DFG::ByteCodeParser::makeSafe):
 177 * dfg/DFGGraph.cpp:
 178 (JSC::DFG::Graph::methodOfGettingAValueProfileFor):
 179 * dfg/DFGJITCompiler.cpp:
 180 (JSC::DFG::JITCompiler::exceptionCheck):
 181 * dfg/DFGSlowPathGenerator.h:
 182 (JSC::DFG::SlowPathGenerator::generate):
 183 * dfg/DFGSpeculativeJIT.cpp:
 184 (JSC::DFG::SpeculativeJIT::addSlowPathGenerator):
 185 (JSC::DFG::SpeculativeJIT::runSlowPathGenerators):
 186 (JSC::DFG::SpeculativeJIT::compileValueAdd):
 187 * dfg/DFGSpeculativeJIT.h:
 188 (JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl):
 189 (JSC::DFG::SpeculativeJIT::silentSpillAllRegisters):
 190 (JSC::DFG::SpeculativeJIT::callOperation):
 191 * ftl/FTLLowerDFGToB3.cpp:
 192 (JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
 193 (JSC::FTL::DFG::LowerDFGToB3::compileStrCat):
 194 (JSC::FTL::DFG::LowerDFGToB3::emitBinarySnippet):
 195 (JSC::FTL::DFG::LowerDFGToB3::emitAddSnippet):
 196 (JSC::FTL::DFG::LowerDFGToB3::emitBinaryBitOpSnippet):
 197 * jit/CCallHelpers.h:
 198 (JSC::CCallHelpers::setupArgumentsWithExecState):
 199 (JSC::CCallHelpers::setupArguments):
 200 * jit/JIT.h:
 201 * jit/JITAddGenerator.cpp:
 202 (JSC::JITAddGenerator::generateInline):
 203 (JSC::JITAddGenerator::generateFastPath):
 204 * jit/JITAddGenerator.h:
 205 (JSC::JITAddGenerator::JITAddGenerator):
 206 (JSC::JITAddGenerator::didEmitFastPath): Deleted.
 207 (JSC::JITAddGenerator::endJumpList): Deleted.
 208 (JSC::JITAddGenerator::slowPathJumpList): Deleted.
 209 * jit/JITArithmetic.cpp:
 210 (JSC::JIT::emit_op_jless):
 211 (JSC::JIT::emitSlow_op_urshift):
 212 (JSC::getOperandTypes):
 213 (JSC::JIT::emit_op_add):
 214 (JSC::JIT::emitSlow_op_add):
 215 (JSC::JIT::emit_op_div):
 216 (JSC::JIT::emit_op_mul):
 217 (JSC::JIT::emitSlow_op_mul):
 218 (JSC::JIT::emit_op_sub):
 219 (JSC::JIT::emitSlow_op_sub):
 220 * jit/JITDivGenerator.cpp:
 221 (JSC::JITDivGenerator::generateFastPath):
 222 * jit/JITDivGenerator.h:
 223 (JSC::JITDivGenerator::JITDivGenerator):
 224 * jit/JITInlines.h:
 225 (JSC::JIT::callOperation):
 226 * jit/JITMathIC.h: Added.
 227 (JSC::JITMathIC::doneLocation):
 228 (JSC::JITMathIC::slowPathStartLocation):
 229 (JSC::JITMathIC::slowPathCallLocation):
 230 (JSC::JITMathIC::generateInline):
 231 (JSC::JITMathIC::generateOutOfLine):
 232 (JSC::JITMathIC::finalizeInlineCode):
 233 * jit/JITMathICForwards.h: Added.
 234 * jit/JITMathICInlineResult.h: Added.
 235 * jit/JITMulGenerator.cpp:
 236 (JSC::JITMulGenerator::generateFastPath):
 237 * jit/JITMulGenerator.h:
 238 (JSC::JITMulGenerator::JITMulGenerator):
 239 * jit/JITOperations.cpp:
 240 * jit/JITOperations.h:
 241 * jit/JITSubGenerator.cpp:
 242 (JSC::JITSubGenerator::generateFastPath):
 243 * jit/JITSubGenerator.h:
 244 (JSC::JITSubGenerator::JITSubGenerator):
 245 * jit/Repatch.cpp:
 246 (JSC::readCallTarget):
 247 (JSC::ftlThunkAwareRepatchCall):
 248 (JSC::tryCacheGetByID):
 249 (JSC::repatchGetByID):
 250 (JSC::appropriateGenericPutByIdFunction):
 251 (JSC::tryCachePutByID):
 252 (JSC::repatchPutByID):
 253 (JSC::tryRepatchIn):
 254 (JSC::repatchIn):
 255 (JSC::linkSlowFor):
 256 (JSC::resetGetByID):
 257 (JSC::resetPutByID):
 258 (JSC::repatchCall): Deleted.
 259 * jit/Repatch.h:
 260 * llint/LLIntData.cpp:
 261 (JSC::LLInt::Data::performAssertions):
 262 * llint/LowLevelInterpreter.asm:
 263 * llint/LowLevelInterpreter32_64.asm:
 264 * llint/LowLevelInterpreter64.asm:
 265 * parser/ResultType.h:
 266 (JSC::ResultType::ResultType):
 267 (JSC::ResultType::isInt32):
 268 (JSC::ResultType::definitelyIsNumber):
 269 (JSC::ResultType::definitelyIsString):
 270 (JSC::ResultType::definitelyIsBoolean):
 271 (JSC::ResultType::mightBeNumber):
 272 (JSC::ResultType::isNotNumber):
 273 (JSC::ResultType::forBitOp):
 274 (JSC::ResultType::bits):
 275 (JSC::OperandTypes::OperandTypes):
 276 * runtime/CommonSlowPaths.cpp:
 277 (JSC::SLOW_PATH_DECL):
 278 (JSC::updateArithProfileForBinaryArithOp):
 279 (JSC::updateResultProfileForBinaryArithOp): Deleted.
 280 * tests/stress/op-add-exceptions.js: Added.
 281 (assert):
 282 (f1):
 283 (f2):
 284 (f3):
 285 (let.oException.valueOf):
 286 (foo):
 287 (ident):
 288 (bar):
 289
12902016-07-20 Michael Saboff <msaboff@apple.com>
2291
3292 CrashOnOverflow in JSC::Yarr::YarrPatternConstructor::setupAlternativeOffsets
203469

Source/JavaScriptCore/CMakeLists.txt

@@set(JavaScriptCore_SOURCES
175175 builtins/BuiltinExecutableCreator.cpp
176176
177177 bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp
 178 bytecode/ArithProfile.cpp
178179 bytecode/ArrayAllocationProfile.cpp
179180 bytecode/ArrayProfile.cpp
180181 bytecode/BytecodeBasicBlock.cpp
203461

Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

12791279 7905BB691D12050E0019FE57 /* InlineAccess.h in Headers */ = {isa = PBXBuildFile; fileRef = 7905BB671D12050E0019FE57 /* InlineAccess.h */; settings = {ATTRIBUTES = (Private, ); }; };
12801280 79160DBD1C8E3EC8008C085A /* ProxyRevoke.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */; };
12811281 79160DBE1C8E3EC8008C085A /* ProxyRevoke.h in Headers */ = {isa = PBXBuildFile; fileRef = 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */; settings = {ATTRIBUTES = (Private, ); }; };
 1282 79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */ = {isa = PBXBuildFile; fileRef = 79233C291D34715700C5A834 /* JITMathIC.h */; settings = {ATTRIBUTES = (Private, ); }; };
12821283 792CB3491C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */; };
12831284 792CB34A1C4EED5C00D13AF3 /* PCToCodeOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
12841285 7964656A1B952FF0003059EE /* GetPutInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 796465681B952FF0003059EE /* GetPutInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
12851286 797E07A91B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 797E07A71B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp */; };
12861287 797E07AA1B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h in Headers */ = {isa = PBXBuildFile; fileRef = 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */; settings = {ATTRIBUTES = (Private, ); }; };
12871288 799EF7C41C56ED96002B0534 /* B3PCToOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
 1289 79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79A228331D35D71E00D8E067 /* ArithProfile.cpp */; };
 1290 79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A228341D35D71E00D8E067 /* ArithProfile.h */; };
 1291 79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */ = {isa = PBXBuildFile; fileRef = 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */; settings = {ATTRIBUTES = (Private, ); }; };
12881292 79B00CBC1C6AB07E0088C65D /* ProxyConstructor.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */; };
12891293 79B00CBD1C6AB07E0088C65D /* ProxyConstructor.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */; settings = {ATTRIBUTES = (Private, ); }; };
12901294 79B00CBE1C6AB07E0088C65D /* ProxyObject.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */; settings = {COMPILER_FLAGS = "-fno-optimize-sibling-calls"; }; };
12911295 79B00CBF1C6AB07E0088C65D /* ProxyObject.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CBB1C6AB07E0088C65D /* ProxyObject.h */; settings = {ATTRIBUTES = (Private, ); }; };
 1296 79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */; settings = {ATTRIBUTES = (Private, ); }; };
12921297 79C4B15D1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79C4B15B1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp */; };
12931298 79C4B15E1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 79C4B15C1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
12941299 79CFC6F01C33B10000C768EA /* LLIntPCRanges.h in Headers */ = {isa = PBXBuildFile; fileRef = 79CFC6EF1C33B10000C768EA /* LLIntPCRanges.h */; settings = {ATTRIBUTES = (Private, ); }; };

34773482 7905BB671D12050E0019FE57 /* InlineAccess.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InlineAccess.h; sourceTree = "<group>"; };
34783483 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyRevoke.cpp; sourceTree = "<group>"; };
34793484 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyRevoke.h; sourceTree = "<group>"; };
 3485 79233C291D34715700C5A834 /* JITMathIC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathIC.h; sourceTree = "<group>"; };
34803486 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PCToCodeOriginMap.cpp; sourceTree = "<group>"; };
34813487 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PCToCodeOriginMap.h; sourceTree = "<group>"; };
34823488 796465681B952FF0003059EE /* GetPutInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GetPutInfo.h; sourceTree = "<group>"; };
34833489 797E07A71B8FCFB9008400BA /* JSGlobalLexicalEnvironment.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSGlobalLexicalEnvironment.cpp; sourceTree = "<group>"; };
34843490 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSGlobalLexicalEnvironment.h; sourceTree = "<group>"; };
34853491 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3PCToOriginMap.h; path = b3/B3PCToOriginMap.h; sourceTree = "<group>"; };
 3492 79A228331D35D71E00D8E067 /* ArithProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArithProfile.cpp; sourceTree = "<group>"; };
 3493 79A228341D35D71E00D8E067 /* ArithProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArithProfile.h; sourceTree = "<group>"; };
 3494 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICForwards.h; sourceTree = "<group>"; };
 3495 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICInlineResult.h; sourceTree = "<group>"; };
34863496 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyConstructor.cpp; sourceTree = "<group>"; };
34873497 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyConstructor.h; sourceTree = "<group>"; };
34883498 79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyObject.cpp; sourceTree = "<group>"; };

50645074 DE5A09FF1BA3AC3E003D4424 /* IntrinsicEmitter.cpp */,
50655075 1429D92D0ED22D7000B89619 /* JIT.cpp */,
50665076 1429D92E0ED22D7000B89619 /* JIT.h */,
 5077 79233C291D34715700C5A834 /* JITMathIC.h */,
 5078 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */,
 5079 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */,
50675080 FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */,
50685081 FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */,
50695082 86A90ECF0EE7D51F00AB350D /* JITArithmetic.cpp */,

65656578 0F8335B51639C1E3001443B5 /* ArrayAllocationProfile.h */,
65666579 0F63945115D07051006A597C /* ArrayProfile.cpp */,
65676580 0F63945215D07051006A597C /* ArrayProfile.h */,
 6581 79A228331D35D71E00D8E067 /* ArithProfile.cpp */,
 6582 79A228341D35D71E00D8E067 /* ArithProfile.h */,
65686583 C2FCAE0C17A9C24E0034C735 /* BytecodeBasicBlock.cpp */,
65696584 C2FCAE0D17A9C24E0034C735 /* BytecodeBasicBlock.h */,
65706585 0F21C27E14BEAA8000ADC64B /* BytecodeConventions.h */,

70777092 A5EA70E819F5B1010098F5EC /* AugmentableInspectorControllerClient.h in Headers */,
70787093 0FEC84FF1BDACDAC0080FF74 /* B3ArgumentRegValue.h in Headers */,
70797094 0FEC85011BDACDAC0080FF74 /* B3BasicBlock.h in Headers */,
 7095 79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */,
70807096 0FEC85021BDACDAC0080FF74 /* B3BasicBlockInlines.h in Headers */,
70817097 0FEC85031BDACDAC0080FF74 /* B3BasicBlockUtils.h in Headers */,
70827098 0FEC85041BDACDAC0080FF74 /* B3BlockWorklist.h in Headers */,

76567672 A7A8AF3A17ADB5F3005AB174 /* Int8Array.h in Headers */,
76577673 BC11667B0E199C05008066DD /* InternalFunction.h in Headers */,
76587674 1429D77C0ED20D7300B89619 /* Interpreter.h in Headers */,
 7675 79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */,
76597676 A1B9E23A1B4E0D6700BC7FED /* IntlCollator.h in Headers */,
76607677 A1B9E23C1B4E0D6700BC7FED /* IntlCollatorConstructor.h in Headers */,
76617678 A18193E31B4E0CDB00FC1029 /* IntlCollatorConstructor.lut.h in Headers */,

77657782 A51007C1187CC3C600B38879 /* JSGlobalObjectInspectorController.h in Headers */,
77667783 A50E4B6418809DD50068A46D /* JSGlobalObjectRuntimeAgent.h in Headers */,
77677784 A503FA2A188F105900110F14 /* JSGlobalObjectScriptDebugServer.h in Headers */,
 7785 79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */,
77687786 A513E5C0185BFACC007E95AD /* JSInjectedScriptHost.h in Headers */,
77697787 A513E5C2185BFACC007E95AD /* JSInjectedScriptHostPrototype.h in Headers */,
77707788 C442CB251A6CDB8C005D3D7C /* JSInputs.json in Headers */,

81908208 0FC8150A14043BF500CFA603 /* WriteBarrierSupport.h in Headers */,
81918209 9688CB160ED12B4E001D649F /* X86Assembler.h in Headers */,
81928210 9959E92E1BD17FA4001AA413 /* xxd.pl in Headers */,
 8211 79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */,
81938212 451539B912DC994500EF7AC4 /* Yarr.h in Headers */,
81948213 86704B8512DBA33700A9FE7B /* YarrInterpreter.h in Headers */,
81958214 86704B8712DBA33700A9FE7B /* YarrJIT.h in Headers */,

92749293 0FEB3ECF16237F6C00AB67AD /* MacroAssembler.cpp in Sources */,
92759294 86C568E011A213EE0007F7F0 /* MacroAssemblerARM.cpp in Sources */,
92769295 FEB137571BB11EF900CD5100 /* MacroAssemblerARM64.cpp in Sources */,
 9296 79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */,
92779297 A729009C17976C6000317298 /* MacroAssemblerARMv7.cpp in Sources */,
92789298 FE68C6381B90DE0B0042BCB3 /* MacroAssemblerPrinter.cpp in Sources */,
92799299 A7A4AE0817973B26005612B1 /* MacroAssemblerX86Common.cpp in Sources */,
203461

Source/JavaScriptCore/bytecode/ArithProfile.cpp

 1/*
 2 * Copyright (C) 2016 Apple Inc. All rights reserved.
 3 *
 4 * Redistribution and use in source and binary forms, with or without
 5 * modification, are permitted provided that the following conditions
 6 * are met:
 7 * 1. Redistributions of source code must retain the above copyright
 8 * notice, this list of conditions and the following disclaimer.
 9 * 2. Redistributions in binary form must reproduce the above copyright
 10 * notice, this list of conditions and the following disclaimer in the
 11 * documentation and/or other materials provided with the distribution.
 12 *
 13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
 14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
 17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
 21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 24 */
 25
 26#include "config.h"
 27#include "ArithProfile.h"
 28
 29#include "CCallHelpers.h"
 30#include "JSCInlines.h"
 31
 32namespace JSC {
 33
 34#if ENABLE(JIT)
 35void ArithProfile::emitDetectNumericness(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)
 36{
 37 if (!shouldEmitSetDouble() && !shouldEmitSetNonNumber())
 38 return;
 39
 40 CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);
 41 CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);
 42 emitSetDouble(jit);
 43 CCallHelpers::Jump done = jit.jump();
 44 notDouble.link(&jit);
 45 emitSetNonNumber(jit);
 46 done.link(&jit);
 47 isInt32.link(&jit);
 48}
 49
 50bool ArithProfile::shouldEmitSetDouble() const
 51{
 52 uint32_t mask = ArithProfile::Int32Overflow | ArithProfile::Int52Overflow | ArithProfile::NegZeroDouble | ArithProfile::NonNegZeroDouble;
 53 return (m_bits & mask) != mask;
 54}
 55
 56void ArithProfile::emitSetDouble(CCallHelpers& jit)
 57{
 58 if (shouldEmitSetDouble())
 59 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int32Overflow | ArithProfile::Int52Overflow | ArithProfile::NegZeroDouble | ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfBits()));
 60}
 61
 62bool ArithProfile::shouldEmitSetNonNumber() const
 63{
 64 uint32_t mask = ArithProfile::NonNumber;
 65 return (m_bits & mask) != mask;
 66}
 67
 68void ArithProfile::emitSetNonNumber(CCallHelpers& jit)
 69{
 70 if (shouldEmitSetNonNumber())
 71 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfBits()));
 72}
 73#endif // ENABLE(JIT)
 74
 75} // namespace JSC
 76
 77namespace WTF {
 78
 79using namespace JSC;
 80
 81void printInternal(PrintStream& out, const ArithProfile& profile)
 82{
 83 const char* separator = "";
 84
 85 if (!profile.didObserveNonInt32()) {
 86 out.print("Int32");
 87 separator = "|";
 88 } else {
 89 if (profile.didObserveNegZeroDouble()) {
 90 out.print(separator, "NegZeroDouble");
 91 separator = "|";
 92 }
 93 if (profile.didObserveNonNegZeroDouble()) {
 94 out.print("NonNegZeroDouble");
 95 separator = "|";
 96 }
 97 if (profile.didObserveNonNumber()) {
 98 out.print("NonNumber");
 99 separator = "|";
 100 }
 101 if (profile.didObserveInt32Overflow()) {
 102 out.print("Int32Overflow");
 103 separator = "|";
 104 }
 105 if (profile.didObserveInt52Overflow()) {
 106 out.print("Int52Overflow");
 107 separator = "|";
 108 }
 109 }
 110 if (profile.tookSpecialFastPath())
 111 out.print(" took special fast path.");
 112}
 113
 114} // namespace WTF
nonexistent

Source/JavaScriptCore/bytecode/ArithProfile.h

 1/*
 2 * Copyright (C) 2016 Apple Inc. All rights reserved.
 3 *
 4 * Redistribution and use in source and binary forms, with or without
 5 * modification, are permitted provided that the following conditions
 6 * are met:
 7 * 1. Redistributions of source code must retain the above copyright
 8 * notice, this list of conditions and the following disclaimer.
 9 * 2. Redistributions in binary form must reproduce the above copyright
 10 * notice, this list of conditions and the following disclaimer in the
 11 * documentation and/or other materials provided with the distribution.
 12 *
 13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
 14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
 17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
 21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 24 */
 25
 26#pragma once
 27
 28#include "GPRInfo.h"
 29#include "JSCJSValue.h"
 30#include "ResultType.h"
 31#include "TagRegistersMode.h"
 32
 33namespace JSC {
 34
 35class CCallHelpers;
 36
 37struct ObservedType {
 38 ObservedType(uint8_t bits = TypeEmpty)
 39 : m_bits(bits)
 40 { }
 41
 42 bool sawInt32() const { return m_bits & TypeInt32; }
 43 bool isOnlyInt32() const { return m_bits == TypeInt32; }
 44 bool sawNumber() const { return m_bits & TypeNumber; }
 45 bool isOnlyNumber() const { return m_bits == TypeNumber; }
 46 bool sawNonNumber() const { return m_bits & TypeNonNumber; }
 47 bool isOnlyNonNumber() const { return m_bits == TypeNonNumber; }
 48 bool isEmpty() const { return !m_bits; }
 49 uint8_t bits() const { return m_bits; }
 50
 51 ObservedType withInt32() const { return ObservedType(m_bits | TypeInt32); }
 52 ObservedType withNumber() const { return ObservedType(m_bits | TypeNumber); }
 53 ObservedType withNonNumber() const { return ObservedType(m_bits | TypeNonNumber); }
 54 ObservedType withoutNonNumber() const { return ObservedType(m_bits & ~TypeNonNumber); }
 55
 56 bool operator==(const ObservedType& other) const { return m_bits == other.m_bits; }
 57
 58 static const uint8_t TypeEmpty = 0x0;
 59 static const uint8_t TypeInt32 = 0x1;
 60 static const uint8_t TypeNumber = 0x02;
 61 static const uint8_t TypeNonNumber = 0x04;
 62
 63private:
 64 uint8_t m_bits;
 65};
 66
 67struct ArithProfile {
 68private:
 69 static const uint32_t numberOfFlagBits = 5;
 70 static const uint32_t rhsResultTypeShift = 5;
 71 static const uint32_t lhsResultTypeShift = 11;
 72 static const uint32_t rhsObservedTypeShift = 17;
 73 static const uint32_t lhsObservedTypeShift = 20;
 74 static const uint32_t bottom6Bits = (1 << 6) - 1;
 75 static const uint32_t bottom3Bits = (1 << 3) - 1;
 76 static const uint32_t clearRhsObservedTypeBits = ~((1 << 17) | (1 << 18) | (1 << 19));
 77 static const uint32_t clearLhsObservedTypeBits = ~((1 << 20) | (1 << 21) | (1 << 22));
 78
 79 static_assert(ResultType::numBitsNeeded == 6, "We make a strong assumption here that ResultType fits in 6 bits.");
 80
 81public:
 82 static const uint32_t specialFastPathBit = 1 << 23;
 83
 84 ArithProfile(ResultType lhs, ResultType rhs)
 85 {
 86 m_bits = (lhs.bits() << lhsResultTypeShift) | (rhs.bits() << rhsResultTypeShift);
 87 ASSERT(lhsResultType().bits() == lhs.bits() && rhsResultType().bits() == rhs.bits());
 88 ASSERT(lhsObservedType().isEmpty());
 89 ASSERT(rhsObservedType().isEmpty());
 90 }
 91 ArithProfile()
 92 { }
 93
 94 static ArithProfile fromInt(uint32_t bits)
 95 {
 96 ArithProfile result;
 97 result.m_bits = bits;
 98 return result;
 99 }
 100
 101 enum ObservedResults {
 102 NonNegZeroDouble = 1 << 0,
 103 NegZeroDouble = 1 << 1,
 104 NonNumber = 1 << 2,
 105 Int32Overflow = 1 << 3,
 106 Int52Overflow = 1 << 4,
 107 };
 108
 109 ResultType lhsResultType() const { return ResultType((m_bits >> lhsResultTypeShift) & bottom6Bits); }
 110 ResultType rhsResultType() const { return ResultType((m_bits >> rhsResultTypeShift) & bottom6Bits); }
 111
 112 ObservedType lhsObservedType() const { return ObservedType((m_bits >> lhsObservedTypeShift) & bottom3Bits); }
 113 ObservedType rhsObservedType() const { return ObservedType((m_bits >> rhsObservedTypeShift) & bottom3Bits); }
 114 void setLhsObservedType(ObservedType type)
 115 {
 116 uint32_t bits = m_bits;
 117 bits &= clearLhsObservedTypeBits;
 118 bits |= type.bits() << lhsObservedTypeShift;
 119 m_bits = bits;
 120 ASSERT(lhsObservedType() == type);
 121 }
 122
 123 void setRhsObservedType(ObservedType type)
 124 {
 125 uint32_t bits = m_bits;
 126 bits &= clearRhsObservedTypeBits;
 127 bits |= type.bits() << rhsObservedTypeShift;
 128 m_bits = bits;
 129 ASSERT(rhsObservedType() == type);
 130 }
 131
 132 bool tookSpecialFastPath() const { return m_bits & specialFastPathBit; }
 133
 134 bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }
 135 bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }
 136 bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }
 137 bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }
 138 bool didObserveNonNumber() const { return hasBits(NonNumber); }
 139 bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }
 140 bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }
 141
 142 void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }
 143 void setObservedNegZeroDouble() { setBit(NegZeroDouble); }
 144 void setObservedNonNumber() { setBit(NonNumber); }
 145 void setObservedInt32Overflow() { setBit(Int32Overflow); }
 146 void setObservedInt52Overflow() { setBit(Int52Overflow); }
 147
 148 void* addressOfBits() { return &m_bits; }
 149
 150 void detectNumericness(JSValue value)
 151 {
 152 if (value.isInt32())
 153 return;
 154 if (value.isNumber()) {
 155 m_bits |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;
 156 return;
 157 }
 158 m_bits |= NonNumber;
 159 }
 160
 161 void lhsSawInt32() { setLhsObservedType(lhsObservedType().withInt32()); }
 162 void lhsSawNumber() { setLhsObservedType(lhsObservedType().withNumber()); }
 163 void lhsSawNonNumber() { setLhsObservedType(lhsObservedType().withNonNumber()); }
 164 void rhsSawInt32() { setRhsObservedType(rhsObservedType().withInt32()); }
 165 void rhsSawNumber() { setRhsObservedType(rhsObservedType().withNumber()); }
 166 void rhsSawNonNumber() { setRhsObservedType(rhsObservedType().withNonNumber()); }
 167
 168 void observeLHSAndRHS(JSValue lhs, JSValue rhs)
 169 {
 170 ArithProfile newProfile = *this;
 171 if (lhs.isNumber()) {
 172 if (lhs.isInt32())
 173 newProfile.lhsSawInt32();
 174 else
 175 newProfile.lhsSawNumber();
 176 } else
 177 newProfile.lhsSawNonNumber();
 178
 179 if (rhs.isNumber()) {
 180 if (rhs.isInt32())
 181 newProfile.rhsSawInt32();
 182 else
 183 newProfile.rhsSawNumber();
 184 } else
 185 newProfile.rhsSawNonNumber();
 186
 187 m_bits = newProfile.bits();
 188 }
 189
 190#if ENABLE(JIT)
 191 // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a
 192 // double. Sets NonNumber if it sees a non-number.
 193 void emitDetectNumericness(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);
 194
 195 // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).
 196 bool shouldEmitSetDouble() const;
 197 void emitSetDouble(CCallHelpers&);
 198
 199 // Sets NonNumber.
 200 void emitSetNonNumber(CCallHelpers&);
 201 bool shouldEmitSetNonNumber() const;
 202#endif // ENABLE(JIT)
 203
 204 uint32_t bits() const { return m_bits; }
 205
 206private:
 207 bool hasBits(int mask) const { return m_bits & mask; }
 208 void setBit(int mask) { m_bits |= mask; }
 209
 210 uint32_t m_bits { 0 }; // We take care to update m_bits only in a single operation. We don't ever store an inconsistent bit representation to it.
 211};
 212
 213} // namespace JSC
 214
 215namespace WTF {
 216
 217void printInternal(PrintStream&, const JSC::ArithProfile&);
 218
 219} // namespace WTF
nonexistent

Source/JavaScriptCore/bytecode/CodeBlock.cpp

3030#include "config.h"
3131#include "CodeBlock.h"
3232
 33#include "ArithProfile.h"
3334#include "BasicBlockLocation.h"
3435#include "BytecodeGenerator.h"
3536#include "BytecodeUseDef.h"

4546#include "InlineCallFrame.h"
4647#include "Interpreter.h"
4748#include "JIT.h"
 49#include "JITMathIC.h"
4850#include "JSCJSValue.h"
4951#include "JSFunction.h"
5052#include "JSLexicalEnvironment.h"

@@void CodeBlock::dumpRareCaseProfile(Prin
769771 out.print(name, profile->m_counter);
770772}
771773
772 void CodeBlock::dumpResultProfile(PrintStream& out, ResultProfile* profile, bool& hasPrintedProfiling)
 774void CodeBlock::dumpArithProfile(PrintStream& out, ArithProfile* profile, bool& hasPrintedProfiling)
773775{
774776 if (!profile)
775777 return;

@@void CodeBlock::dumpBytecode(
17701772 dumpRareCaseProfile(out, "rare case: ", rareCaseProfileForBytecodeOffset(location), hasPrintedProfiling);
17711773 {
17721774 ConcurrentJITLocker locker(m_lock);
1773  dumpResultProfile(out, resultProfileForBytecodeOffset(locker, location), hasPrintedProfiling);
 1775 dumpArithProfile(out, arithProfileForBytecodeOffset(location), hasPrintedProfiling);
17741776 }
17751777
17761778#if ENABLE(DFG_JIT)

@@StructureStubInfo* CodeBlock::addStubInf
30033005 return m_stubInfos.add(accessType);
30043006}
30053007
 3008JITAddIC* CodeBlock::addJITAddIC()
 3009{
 3010 return m_addICs.add();
 3011}
 3012
30063013StructureStubInfo* CodeBlock::findStubInfo(CodeOrigin codeOrigin)
30073014{
30083015 for (StructureStubInfo* stubInfo : m_stubInfos) {

@@void CodeBlock::resetJITData()
30493056 // We can clear this because the DFG's queries to these data structures are guarded by whether
30503057 // there is JIT code.
30513058 m_rareCaseProfiles.clear();
3052 
3053  // We can clear these because the DFG only accesses members of this data structure when
3054  // holding the lock or after querying whether we have JIT code.
3055  m_resultProfiles.clear();
3056  m_bytecodeOffsetToResultProfileIndexMap = nullptr;
30573059}
30583060#endif
30593061

@@void CodeBlock::shrinkToFit(ShrinkMode s
32833285 ConcurrentJITLocker locker(m_lock);
32843286
32853287 m_rareCaseProfiles.shrinkToFit();
3286  m_resultProfiles.shrinkToFit();
32873288
32883289 if (shrinkMode == EarlyShrink) {
32893290 m_constantRegisters.shrinkToFit();

@@void CodeBlock::dumpValueProfiles()
41594160 RareCaseProfile* profile = rareCaseProfile(i);
41604161 dataLogF(" bc = %d: %u\n", profile->m_bytecodeOffset, profile->m_counter);
41614162 }
4162  dataLog("ResultProfile for ", *this, ":\n");
4163  for (unsigned i = 0; i < numberOfResultProfiles(); ++i) {
4164  const ResultProfile& profile = *resultProfile(i);
4165  dataLog(" bc = ", profile.bytecodeOffset(), ": ", profile, "\n");
4166  }
41674163}
41684164#endif // ENABLE(VERBOSE_VALUE_PROFILE)
41694165

@@unsigned CodeBlock::rareCaseProfileCount
43664362 return 0;
43674363}
43684364
4369 ResultProfile* CodeBlock::resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset)
 4365ArithProfile* CodeBlock::arithProfileForBytecodeOffset(int bytecodeOffset)
43704366{
4371  if (!m_bytecodeOffsetToResultProfileIndexMap)
4372  return nullptr;
4373  auto iterator = m_bytecodeOffsetToResultProfileIndexMap->find(bytecodeOffset);
4374  if (iterator == m_bytecodeOffsetToResultProfileIndexMap->end())
 4367 auto opcodeID = vm()->interpreter->getOpcodeID(instructions()[bytecodeOffset].u.opcode);
 4368 switch (opcodeID) {
 4369 case op_bitor:
 4370 case op_bitand:
 4371 case op_bitxor:
 4372 case op_add:
 4373 case op_mul:
 4374 case op_sub:
 4375 case op_div:
 4376 break;
 4377 default:
43754378 return nullptr;
4376  return &m_resultProfiles[iterator->value];
4377 }
 4379 }
43784380
4379 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset)
4380 {
4381  ConcurrentJITLocker locker(m_lock);
4382  return specialFastCaseProfileCountForBytecodeOffset(locker, bytecodeOffset);
 4381 Instruction* instruction = &instructions()[bytecodeOffset + 4];
 4382 return bitwise_cast<ArithProfile*>(&instruction->u.operand);
43834383}
43844384
4385 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker& locker, int bytecodeOffset)
 4385ArithProfile& CodeBlock::arithProfileForPC(Instruction* pc)
43864386{
4387  ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
4388  if (!profile)
4389  return 0;
4390  return profile->specialFastPathCount();
 4387 if (!ASSERT_DISABLED) {
 4388 ASSERT(pc >= instructions().begin() && pc < instructions().end());
 4389 auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode);
 4390 switch (opcodeID) {
 4391 case op_bitor:
 4392 case op_bitand:
 4393 case op_bitxor:
 4394 case op_add:
 4395 case op_mul:
 4396 case op_sub:
 4397 case op_div:
 4398 break;
 4399 default:
 4400 ASSERT_NOT_REACHED();
 4401 }
 4402 }
 4403
 4404 return *bitwise_cast<ArithProfile*>(&pc[4].u.operand);
43914405}
43924406
43934407bool CodeBlock::couldTakeSpecialFastCase(int bytecodeOffset)
43944408{
43954409 if (!hasBaselineJITProfiling())
43964410 return false;
4397  unsigned specialFastCaseCount = specialFastCaseProfileCountForBytecodeOffset(bytecodeOffset);
4398  return specialFastCaseCount >= Options::couldTakeSlowCaseMinimumCount();
4399 }
4400 
4401 ResultProfile* CodeBlock::ensureResultProfile(int bytecodeOffset)
4402 {
4403  ConcurrentJITLocker locker(m_lock);
4404  return ensureResultProfile(locker, bytecodeOffset);
4405 }
4406 
4407 ResultProfile* CodeBlock::ensureResultProfile(const ConcurrentJITLocker& locker, int bytecodeOffset)
4408 {
4409  ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
4410  if (!profile) {
4411  m_resultProfiles.append(ResultProfile(bytecodeOffset));
4412  profile = &m_resultProfiles.last();
4413  ASSERT(&m_resultProfiles.last() == &m_resultProfiles[m_resultProfiles.size() - 1]);
4414  if (!m_bytecodeOffsetToResultProfileIndexMap)
4415  m_bytecodeOffsetToResultProfileIndexMap = std::make_unique<BytecodeOffsetToResultProfileIndexMap>();
4416  m_bytecodeOffsetToResultProfileIndexMap->add(bytecodeOffset, m_resultProfiles.size() - 1);
4417  }
4418  return profile;
 4411 ArithProfile* profile = arithProfileForBytecodeOffset(bytecodeOffset);
 4412 if (!profile)
 4413 return false;
 4414 return profile->tookSpecialFastPath();
44194415}
44204416
44214417#if ENABLE(JIT)
203461

Source/JavaScriptCore/bytecode/CodeBlock.h

5151#include "HandlerInfo.h"
5252#include "Instruction.h"
5353#include "JITCode.h"
 54#include "JITMathICForwards.h"
5455#include "JITWriteBarrier.h"
5556#include "JSCell.h"
5657#include "JSGlobalObject.h"

7980namespace JSC {
8081
8182class ExecState;
 83class JITAddGenerator;
8284class JSModuleEnvironment;
8385class LLIntOffsetsExtractor;
8486class PCToCodeOriginMap;

@@class TypeLocation;
8890
8991enum class AccessType : int8_t;
9092
 93struct ArithProfile;
 94
9195typedef HashMap<CodeOrigin, StructureStubInfo*, CodeOriginApproximateHash> StubInfoMap;
9296
9397enum ReoptimizationMode { DontCountReoptimization, CountReoptimization };

@@public:
245249
246250#if ENABLE(JIT)
247251 StructureStubInfo* addStubInfo(AccessType);
 252 JITAddIC* addJITAddIC();
248253 Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); }
249254 Bag<StructureStubInfo>::iterator stubInfoEnd() { return m_stubInfos.end(); }
250255

@@public:
440445 return value >= Options::couldTakeSlowCaseMinimumCount();
441446 }
442447
443  ResultProfile* ensureResultProfile(int bytecodeOffset);
444  ResultProfile* ensureResultProfile(const ConcurrentJITLocker&, int bytecodeOffset);
445  unsigned numberOfResultProfiles() { return m_resultProfiles.size(); }
446  ResultProfile* resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
447 
448  unsigned specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
449  unsigned specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset);
 448 ArithProfile* arithProfileForBytecodeOffset(int bytecodeOffset);
 449 ArithProfile& arithProfileForPC(Instruction*);
450450
451451 bool couldTakeSpecialFastCase(int bytecodeOffset);
452452

@@private:
957957 void dumpValueProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
958958 void dumpArrayProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
959959 void dumpRareCaseProfile(PrintStream&, const char* name, RareCaseProfile*, bool& hasPrintedProfiling);
960  void dumpResultProfile(PrintStream&, ResultProfile*, bool& hasPrintedProfiling);
 960 void dumpArithProfile(PrintStream&, ArithProfile*, bool& hasPrintedProfiling);
961961
962962 bool shouldVisitStrongly();
963963 bool shouldJettisonDueToWeakReference();

@@private:
10131013#if ENABLE(JIT)
10141014 std::unique_ptr<RegisterAtOffsetList> m_calleeSaveRegisters;
10151015 Bag<StructureStubInfo> m_stubInfos;
 1016 Bag<JITAddIC> m_addICs;
10161017 Bag<ByValInfo> m_byValInfos;
10171018 Bag<CallLinkInfo> m_callLinkInfos;
10181019 SentinelLinkedList<CallLinkInfo, BasicRawSentinelNode<CallLinkInfo>> m_incomingCalls;

@@private:
10291030 RefCountedArray<ValueProfile> m_argumentValueProfiles;
10301031 RefCountedArray<ValueProfile> m_valueProfiles;
10311032 SegmentedVector<RareCaseProfile, 8> m_rareCaseProfiles;
1032  SegmentedVector<ResultProfile, 8> m_resultProfiles;
1033  typedef HashMap<unsigned, unsigned, IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> BytecodeOffsetToResultProfileIndexMap;
1034  std::unique_ptr<BytecodeOffsetToResultProfileIndexMap> m_bytecodeOffsetToResultProfileIndexMap;
10351033 RefCountedArray<ArrayAllocationProfile> m_arrayAllocationProfiles;
10361034 ArrayProfileVector m_arrayProfiles;
10371035 RefCountedArray<ObjectAllocationProfile> m_objectAllocationProfiles;
203461

Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.cpp

2828
2929#if ENABLE(DFG_JIT)
3030
 31#include "ArithProfile.h"
3132#include "CCallHelpers.h"
3233#include "CodeBlock.h"
3334#include "JSCInlines.h"

@@void MethodOfGettingAValueProfile::emitR
6566 return;
6667 }
6768
68  case ResultProfileReady: {
69  u.resultProfile->emitDetectNumericness(jit, regs, DoNotHaveTagRegisters);
 69 case ArithProfileReady: {
 70 u.arithProfile->emitDetectNumericness(jit, regs, DoNotHaveTagRegisters);
7071 return;
7172 } }
7273
203461

Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.h

@@namespace JSC {
4040class CCallHelpers;
4141class CodeBlock;
4242class LazyOperandValueProfileKey;
43 struct ResultProfile;
 43struct ArithProfile;
4444struct ValueProfile;
4545
4646class MethodOfGettingAValueProfile {

@@public:
5959 m_kind = None;
6060 }
6161
62  MethodOfGettingAValueProfile(ResultProfile* profile)
 62 MethodOfGettingAValueProfile(ArithProfile* profile)
6363 {
6464 if (profile) {
65  m_kind = ResultProfileReady;
66  u.resultProfile = profile;
 65 m_kind = ArithProfileReady;
 66 u.arithProfile = profile;
6767 } else
6868 m_kind = None;
6969 }

@@private:
7979 enum Kind {
8080 None,
8181 Ready,
82  ResultProfileReady,
 82 ArithProfileReady,
8383 LazyOperand
8484 };
8585
8686 Kind m_kind;
8787 union {
8888 ValueProfile* profile;
89  ResultProfile* resultProfile;
 89 ArithProfile* arithProfile;
9090 struct {
9191 CodeBlock* codeBlock;
9292 unsigned bytecodeOffset;
203461

Source/JavaScriptCore/bytecode/ValueProfile.cpp

3131
3232namespace JSC {
3333
34 #if ENABLE(JIT)
35 void ResultProfile::emitDetectNumericness(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)
36 {
37  CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);
38  CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);
39  // FIXME: We could be more precise here.
40  emitSetDouble(jit);
41  CCallHelpers::Jump done = jit.jump();
42  notDouble.link(&jit);
43  emitSetNonNumber(jit);
44  done.link(&jit);
45  isInt32.link(&jit);
46 }
47 
48 void ResultProfile::emitSetDouble(CCallHelpers& jit)
49 {
50  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int32Overflow | ResultProfile::Int52Overflow | ResultProfile::NegZeroDouble | ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfFlags()));
51 }
52 
53 void ResultProfile::emitSetNonNumber(CCallHelpers& jit)
54 {
55  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfFlags()));
56 }
57 #endif // ENABLE(JIT)
58 
5934} // namespace JSC
60 
61 namespace WTF {
62 
63 using namespace JSC;
64 
65 void printInternal(PrintStream& out, const ResultProfile& profile)
66 {
67  const char* separator = "";
68 
69  if (!profile.didObserveNonInt32()) {
70  out.print("Int32");
71  separator = "|";
72  } else {
73  if (profile.didObserveNegZeroDouble()) {
74  out.print(separator, "NegZeroDouble");
75  separator = "|";
76  }
77  if (profile.didObserveNonNegZeroDouble()) {
78  out.print("NonNegZeroDouble");
79  separator = "|";
80  }
81  if (profile.didObserveNonNumber()) {
82  out.print("NonNumber");
83  separator = "|";
84  }
85  if (profile.didObserveInt32Overflow()) {
86  out.print("Int32Overflow");
87  separator = "|";
88  }
89  if (profile.didObserveInt52Overflow()) {
90  out.print("Int52Overflow");
91  separator = "|";
92  }
93  }
94  if (profile.specialFastPathCount()) {
95  out.print(" special fast path: ");
96  out.print(profile.specialFastPathCount());
97  }
98 }
99 
100 } // namespace WTF
203461

Source/JavaScriptCore/bytecode/ValueProfile.h

@@inline int getRareCaseProfileBytecodeOff
209209 return rareCaseProfile->m_bytecodeOffset;
210210}
211211
212 struct ResultProfile {
213 private:
214  static const int numberOfFlagBits = 5;
215 
216 public:
217  ResultProfile(int bytecodeOffset)
218  : m_bytecodeOffsetAndFlags(bytecodeOffset << numberOfFlagBits)
219  {
220  ASSERT(((bytecodeOffset << numberOfFlagBits) >> numberOfFlagBits) == bytecodeOffset);
221  }
222 
223  enum ObservedResults {
224  NonNegZeroDouble = 1 << 0,
225  NegZeroDouble = 1 << 1,
226  NonNumber = 1 << 2,
227  Int32Overflow = 1 << 3,
228  Int52Overflow = 1 << 4,
229  };
230 
231  int bytecodeOffset() const { return m_bytecodeOffsetAndFlags >> numberOfFlagBits; }
232  unsigned specialFastPathCount() const { return m_specialFastPathCount; }
233 
234  bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }
235  bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }
236  bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }
237  bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }
238  bool didObserveNonNumber() const { return hasBits(NonNumber); }
239  bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }
240  bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }
241 
242  void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }
243  void setObservedNegZeroDouble() { setBit(NegZeroDouble); }
244  void setObservedNonNumber() { setBit(NonNumber); }
245  void setObservedInt32Overflow() { setBit(Int32Overflow); }
246  void setObservedInt52Overflow() { setBit(Int52Overflow); }
247 
248  void* addressOfFlags() { return &m_bytecodeOffsetAndFlags; }
249  void* addressOfSpecialFastPathCount() { return &m_specialFastPathCount; }
250 
251  void detectNumericness(JSValue value)
252  {
253  if (value.isInt32())
254  return;
255  if (value.isNumber()) {
256  m_bytecodeOffsetAndFlags |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;
257  return;
258  }
259  m_bytecodeOffsetAndFlags |= NonNumber;
260  }
261 
262 #if ENABLE(JIT)
263  // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a
264  // double. Sets NonNumber if it sees a non-number.
265  void emitDetectNumericness(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);
266 
267  // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).
268  void emitSetDouble(CCallHelpers&);
269 
270  // Sets NonNumber.
271  void emitSetNonNumber(CCallHelpers&);
272 #endif // ENABLE(JIT)
273 
274 private:
275  bool hasBits(int mask) const { return m_bytecodeOffsetAndFlags & mask; }
276  void setBit(int mask) { m_bytecodeOffsetAndFlags |= mask; }
277 
278  int m_bytecodeOffsetAndFlags;
279  unsigned m_specialFastPathCount { 0 };
280 };
281 
282 inline int getResultProfileBytecodeOffset(ResultProfile* profile)
283 {
284  return profile->bytecodeOffset();
285 }
286 
287212} // namespace JSC
288213
289 namespace WTF {
290 
291 void printInternal(PrintStream&, const JSC::ResultProfile&);
292 
293 } // namespace WTF
294 
295214#endif // ValueProfile_h
296 
203461

Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp

3131#include "config.h"
3232#include "BytecodeGenerator.h"
3333
 34#include "ArithProfile.h"
3435#include "BuiltinExecutables.h"
3536#include "BytecodeLivenessAnalysis.h"
3637#include "Interpreter.h"

@@RegisterID* BytecodeGenerator::emitBinar
16001601
16011602 if (opcodeID == op_bitor || opcodeID == op_bitand || opcodeID == op_bitxor ||
16021603 opcodeID == op_add || opcodeID == op_mul || opcodeID == op_sub || opcodeID == op_div)
1603  instructions().append(types.toInt());
 1604 instructions().append(ArithProfile(types.first(), types.second()).bits());
16041605
16051606 return dst;
16061607}
203461

Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

2828
2929#if ENABLE(DFG_JIT)
3030
 31#include "ArithProfile.h"
3132#include "ArrayConstructor.h"
3233#include "BasicBlockLocation.h"
3334#include "CallLinkStatus.h"

@@private:
914915 return node;
915916
916917 {
917  ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
918  ResultProfile* resultProfile = m_inlineStackTop->m_profiledBlock->resultProfileForBytecodeOffset(locker, m_currentIndex);
919  if (resultProfile) {
 918 ArithProfile* arithProfile = m_inlineStackTop->m_profiledBlock->arithProfileForBytecodeOffset(m_currentIndex);
 919 if (arithProfile) {
920920 switch (node->op()) {
921921 case ArithAdd:
922922 case ArithSub:
923923 case ValueAdd:
924  if (resultProfile->didObserveDouble())
 924 if (arithProfile->didObserveDouble())
925925 node->mergeFlags(NodeMayHaveDoubleResult);
926  if (resultProfile->didObserveNonNumber())
 926 if (arithProfile->didObserveNonNumber())
927927 node->mergeFlags(NodeMayHaveNonNumberResult);
928928 break;
929929
930930 case ArithMul: {
931  if (resultProfile->didObserveInt52Overflow())
 931 if (arithProfile->didObserveInt52Overflow())
932932 node->mergeFlags(NodeMayOverflowInt52);
933  if (resultProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
 933 if (arithProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
934934 node->mergeFlags(NodeMayOverflowInt32InBaseline);
935  if (resultProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
 935 if (arithProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
936936 node->mergeFlags(NodeMayNegZeroInBaseline);
937  if (resultProfile->didObserveDouble())
 937 if (arithProfile->didObserveDouble())
938938 node->mergeFlags(NodeMayHaveDoubleResult);
939  if (resultProfile->didObserveNonNumber())
 939 if (arithProfile->didObserveNonNumber())
940940 node->mergeFlags(NodeMayHaveNonNumberResult);
941941 break;
942942 }
203461

Source/JavaScriptCore/dfg/DFGGraph.cpp

@@MethodOfGettingAValueProfile Graph::meth
15481548 return profiledBlock->valueProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
15491549
15501550 {
1551  ConcurrentJITLocker locker(profiledBlock->m_lock);
15521551 if (profiledBlock->hasBaselineJITProfiling()) {
1553  if (ResultProfile* result = profiledBlock->resultProfileForBytecodeOffset(locker, node->origin.semantic.bytecodeIndex))
 1552 if (ArithProfile* result = profiledBlock->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex))
15541553 return result;
15551554 }
15561555 }
203461

Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

@@void JITCompiler::exceptionCheck()
604604 HandlerInfo* exceptionHandler;
605605 bool willCatchException = m_graph.willCatchExceptionInMachineFrame(m_speculative->m_currentNode->origin.forExit, opCatchOrigin, exceptionHandler);
606606 if (willCatchException) {
607  unsigned streamIndex = m_speculative->m_outOfLineStreamIndex != UINT_MAX ? m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
 607 unsigned streamIndex = m_speculative->m_outOfLineStreamIndex ? *m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
608608 MacroAssembler::Jump hadException = emitNonPatchableExceptionCheck();
609609 // We assume here that this is called after callOpeartion()/appendCall() is called.
610610 appendExceptionHandlingOSRExit(ExceptionCheck, streamIndex, opCatchOrigin, exceptionHandler, m_jitCode->common.lastCallSite(), hadException);
203461

Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h

@@public:
5252 jit->m_outOfLineStreamIndex = m_streamIndex;
5353 jit->m_origin = m_origin;
5454 generateInternal(jit);
55  jit->m_outOfLineStreamIndex = UINT_MAX;
 55 jit->m_outOfLineStreamIndex = Nullopt;
5656 if (!ASSERT_DISABLED)
5757 jit->m_jit.abortWithReason(DFGSlowPathGeneratorFellThrough);
5858 }
203461

Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

4444#include "JITBitXorGenerator.h"
4545#include "JITDivGenerator.h"
4646#include "JITLeftShiftGenerator.h"
 47#include "JITMathIC.h"
4748#include "JITMulGenerator.h"
4849#include "JITRightShiftGenerator.h"
4950#include "JITSubGenerator.h"

5657#include "ScopedArguments.h"
5758#include "ScratchRegisterAllocator.h"
5859#include "WriteBarrierBuffer.h"
 60#include <wtf/Box.h>
5961#include <wtf/MathExtras.h>
6062
6163namespace JSC { namespace DFG {

@@void SpeculativeJIT::addSlowPathGenerato
362364
363365void SpeculativeJIT::addSlowPathGenerator(std::function<void()> lambda)
364366{
365  m_slowPathLambdas.append(std::make_pair(lambda, m_currentNode));
 367 m_slowPathLambdas.append(SlowPathLambda{ lambda, m_currentNode, static_cast<unsigned>(m_stream->size()) });
366368}
367369
368370void SpeculativeJIT::runSlowPathGenerators(PCToCodeOriginMapBuilder& pcToCodeOriginMapBuilder)

@@void SpeculativeJIT::runSlowPathGenerato
371373 pcToCodeOriginMapBuilder.appendItem(m_jit.label(), slowPathGenerator->origin().semantic);
372374 slowPathGenerator->generate(this);
373375 }
374  for (auto& generatorPair : m_slowPathLambdas) {
375  Node* currentNode = generatorPair.second;
 376 for (auto& slowPathLambda : m_slowPathLambdas) {
 377 Node* currentNode = slowPathLambda.currentNode;
376378 m_currentNode = currentNode;
 379 m_outOfLineStreamIndex = slowPathLambda.streamIndex;
377380 pcToCodeOriginMapBuilder.appendItem(m_jit.label(), currentNode->origin.semantic);
378  generatorPair.first();
 381 slowPathLambda.generator();
 382 m_outOfLineStreamIndex = Nullopt;
379383 }
380384}
381385

@@void SpeculativeJIT::compileValueAdd(Nod
33793383 rightRegs = right->jsValueRegs();
33803384 }
33813385
3382  JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
3383  leftFPR, rightFPR, scratchGPR, scratchFPR);
3384  gen.generateFastPath(m_jit);
 3386 JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC();
 3387 Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
 3388 ArithProfile* arithProfile = m_jit.graph().baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
 3389 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile);
 3390
 3391 bool generatedInline = addIC->generateInline(m_jit, *addICGenerationState);
 3392
 3393 if (generatedInline) {
 3394 ASSERT(!addICGenerationState->slowPathJumps.empty());
 3395
 3396 Vector<SilentRegisterSavePlan, 4> savePlans;
 3397 silentSpillAllRegistersImpl(false, savePlans, resultRegs);
 3398
 3399 auto done = m_jit.label();
 3400
 3401 addSlowPathGenerator([=] () {
 3402 addICGenerationState->slowPathJumps.link(&m_jit);
 3403 addICGenerationState->slowPathStart = m_jit.label();
 3404
 3405 silentSpill(savePlans);
 3406
 3407 auto innerLeftRegs = leftRegs;
 3408 auto innerRightRegs = rightRegs;
 3409 if (leftOperand.isConst()) {
 3410 innerLeftRegs = resultRegs;
 3411 m_jit.moveValue(leftChild->asJSValue(), innerLeftRegs);
 3412 } else if (rightOperand.isConst()) {
 3413 innerRightRegs = resultRegs;
 3414 m_jit.moveValue(rightChild->asJSValue(), innerRightRegs);
 3415 }
33853416
3386  ASSERT(gen.didEmitFastPath());
3387  gen.endJumpList().append(m_jit.jump());
 3417 if (addICGenerationState->shouldSlowPathRepatch)
 3418 addICGenerationState->slowPathCall = callOperation(operationValueAddOptimize, resultRegs, innerLeftRegs, innerRightRegs, addIC);
 3419 else
 3420 addICGenerationState->slowPathCall = callOperation(operationValueAdd, resultRegs, innerLeftRegs, innerRightRegs);
33883421
3389  gen.slowPathJumpList().link(&m_jit);
 3422 silentFill(savePlans);
 3423 m_jit.exceptionCheck();
 3424 m_jit.jump().linkTo(done, &m_jit);
33903425
3391  silentSpillAllRegisters(resultRegs);
 3426 m_jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
 3427 addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
 3428 });
 3429 });
 3430 } else {
 3431 if (leftOperand.isConst()) {
 3432 left = JSValueOperand(this, leftChild);
 3433 leftRegs = left->jsValueRegs();
 3434 } else if (rightOperand.isConst()) {
 3435 right = JSValueOperand(this, rightChild);
 3436 rightRegs = right->jsValueRegs();
 3437 }
33923438
3393  if (leftOperand.isConst()) {
3394  leftRegs = resultRegs;
3395  m_jit.moveValue(leftChild->asJSValue(), leftRegs);
3396  } else if (rightOperand.isConst()) {
3397  rightRegs = resultRegs;
3398  m_jit.moveValue(rightChild->asJSValue(), rightRegs);
 3439 flushRegisters();
 3440 callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
 3441 m_jit.exceptionCheck();
33993442 }
34003443
3401  callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
3402 
3403  silentFillAllRegisters(resultRegs);
3404  m_jit.exceptionCheck();
3405 
3406  gen.endJumpList().link(&m_jit);
34073444 jsValueResult(resultRegs, node);
34083445 return;
34093446}
203461

Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

@@public:
388388 {
389389 silentSpillAllRegistersImpl(doSpill, plans, InvalidGPRReg, InvalidGPRReg, exclude);
390390 }
391 #if USE(JSVALUE32_64)
392391 template<typename CollectionType>
393392 void silentSpillAllRegistersImpl(bool doSpill, CollectionType& plans, JSValueRegs exclude)
394393 {
 394#if USE(JSVALUE32_64)
395395 silentSpillAllRegistersImpl(doSpill, plans, exclude.tagGPR(), exclude.payloadGPR());
396  }
 396#else
 397 silentSpillAllRegistersImpl(doSpill, plans, exclude.gpr());
397398#endif
 399 }
398400
399401 void silentSpillAllRegisters(GPRReg exclude, GPRReg exclude2 = InvalidGPRReg, FPRReg fprExclude = InvalidFPRReg)
400402 {

@@public:
12851287
12861288
12871289#if USE(JSVALUE64)
 1290
 1291 JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
 1292 {
 1293 m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), TrustedImmPtr(addIC));
 1294 return appendCallSetResult(operation, result.gpr());
 1295 }
 1296
12881297 JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg result, GPRReg arg1, GPRReg arg2, UniquedStringImpl* uid)
12891298 {
12901299 m_jit.setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(uid));

@@public:
17021711 }
17031712#else // USE(JSVALUE32_64)
17041713
 1714 JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
 1715 {
 1716 m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), TrustedImmPtr(addIC));
 1717 return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
 1718 }
 1719
17051720 JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg resultTag, GPRReg resultPayload, GPRReg arg1Tag, GPRReg arg1Payload, GPRReg arg2Tag, GPRReg arg2Payload, UniquedStringImpl* uid)
17061721 {
17071722 m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1Payload, arg1Tag, arg2Payload, arg2Tag, TrustedImmPtr(uid));

@@public:
28312846 MinifiedGraph* m_minifiedGraph;
28322847
28332848 Vector<std::unique_ptr<SlowPathGenerator>, 8> m_slowPathGenerators;
2834  Vector<std::pair<std::function<void()>, Node*>, 8> m_slowPathLambdas;
 2849 struct SlowPathLambda {
 2850 std::function<void()> generator;
 2851 Node* currentNode;
 2852 unsigned streamIndex;
 2853 };
 2854 Vector<SlowPathLambda> m_slowPathLambdas;
28352855 Vector<SilentRegisterSavePlan> m_plans;
2836  unsigned m_outOfLineStreamIndex { UINT_MAX };
 2856 Optional<unsigned> m_outOfLineStreamIndex;
28372857};
28382858
28392859
203461

Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

6262#include "JITDivGenerator.h"
6363#include "JITInlineCacheGenerator.h"
6464#include "JITLeftShiftGenerator.h"
 65#include "JITMathIC.h"
6566#include "JITMulGenerator.h"
6667#include "JITRightShiftGenerator.h"
6768#include "JITSubGenerator.h"

@@private:
15381539
15391540 void compileValueAdd()
15401541 {
1541  emitBinarySnippet<JITAddGenerator>(operationValueAdd);
 1542 emitAddSnippet();
15421543 }
15431544
15441545 void compileStrCat()

@@private:
81508151 setJSValue(patchpoint);
81518152 }
81528153
 8154 void emitAddSnippet()
 8155 {
 8156 Node* node = m_node;
 8157
 8158 LValue left = lowJSValue(node->child1());
 8159 LValue right = lowJSValue(node->child2());
 8160
 8161 SnippetOperand leftOperand(m_state.forNode(node->child1()).resultType());
 8162 SnippetOperand rightOperand(m_state.forNode(node->child2()).resultType());
 8163
 8164 PatchpointValue* patchpoint = m_out.patchpoint(Int64);
 8165 patchpoint->appendSomeRegister(left);
 8166 patchpoint->appendSomeRegister(right);
 8167 patchpoint->append(m_tagMask, ValueRep::lateReg(GPRInfo::tagMaskRegister));
 8168 patchpoint->append(m_tagTypeNumber, ValueRep::lateReg(GPRInfo::tagTypeNumberRegister));
 8169 RefPtr<PatchpointExceptionHandle> exceptionHandle =
 8170 preparePatchpointForExceptions(patchpoint);
 8171 patchpoint->numGPScratchRegisters = 1;
 8172 patchpoint->numFPScratchRegisters = 2;
 8173 patchpoint->clobber(RegisterSet::macroScratchRegisters());
 8174 State* state = &m_ftlState;
 8175 patchpoint->setGenerator(
 8176 [=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
 8177 AllowMacroScratchRegisterUsage allowScratch(jit);
 8178
 8179 Box<CCallHelpers::JumpList> exceptions =
 8180 exceptionHandle->scheduleExitCreation(params)->jumps(jit);
 8181
 8182 JITAddIC* addIC = jit.codeBlock()->addJITAddIC();
 8183 Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
 8184 ArithProfile* arithProfile = state->graph.baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
 8185 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()),
 8186 JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), params.fpScratch(0),
 8187 params.fpScratch(1), params.gpScratch(0), InvalidFPRReg, arithProfile);
 8188
 8189 bool generatedInline = addIC->generateInline(jit, *addICGenerationState);
 8190
 8191 if (generatedInline) {
 8192 ASSERT(!addICGenerationState->slowPathJumps.empty());
 8193 auto done = jit.label();
 8194 params.addLatePath([=] (CCallHelpers& jit) {
 8195 AllowMacroScratchRegisterUsage allowScratch(jit);
 8196 addICGenerationState->slowPathJumps.link(&jit);
 8197 addICGenerationState->slowPathStart = jit.label();
 8198
 8199 if (addICGenerationState->shouldSlowPathRepatch) {
 8200 SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
 8201 operationValueAddOptimize, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(addIC));
 8202 addICGenerationState->slowPathCall = call.call();
 8203 } else {
 8204 SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic,
 8205 exceptions.get(), operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
 8206 addICGenerationState->slowPathCall = call.call();
 8207 }
 8208 jit.jump().linkTo(done, &jit);
 8209
 8210 jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
 8211 addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
 8212 });
 8213 });
 8214 } else {
 8215 callOperation(
 8216 *state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
 8217 operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
 8218 }
 8219 });
 8220
 8221 setJSValue(patchpoint);
 8222 }
 8223
81538224 template<typename BinaryBitOpGenerator>
81548225 void emitBinaryBitOpSnippet(J_JITOperation_EJJ slowPathFunction)
81558226 {
203461

Source/JavaScriptCore/jit/CCallHelpers.h

@@public:
826826 addCallArgument(arg6);
827827 }
828828
 829 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
 830 {
 831 resetCallArguments();
 832 addCallArgument(GPRInfo::callFrameRegister);
 833 addCallArgument(arg1);
 834 addCallArgument(arg2);
 835 addCallArgument(arg3);
 836 addCallArgument(arg4);
 837 addCallArgument(arg5);
 838 addCallArgument(arg6);
 839 }
 840
829841 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, GPRReg arg5, GPRReg arg6, TrustedImmPtr arg7)
830842 {
831843 resetCallArguments();

@@public:
15941606 setupArgumentsWithExecState(arg1, arg2, arg3);
15951607 }
15961608
 1609 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
 1610 {
 1611 poke(arg6, POKE_ARGUMENT_OFFSET + 2);
 1612 poke(arg5, POKE_ARGUMENT_OFFSET + 1);
 1613 poke(arg4, POKE_ARGUMENT_OFFSET);
 1614 setupArgumentsWithExecState(arg1, arg2, arg3);
 1615 }
 1616
15971617 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5)
15981618 {
15991619 poke(arg5, POKE_ARGUMENT_OFFSET + 1);

@@public:
20992119 move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
21002120 }
21012121
 2122 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
 2123 {
 2124 setupTwoStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR2>(arg1, arg2);
 2125 move(arg3, GPRInfo::argumentGPR3);
 2126 move(arg4, GPRInfo::argumentGPR4);
 2127 move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
 2128 }
 2129
21022130 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5)
21032131 {
21042132 setupThreeStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR4, GPRInfo::argumentGPR5>(arg1, arg4, arg5);

@@public:
22072235 }
22082236#endif
22092237
 2238 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2)
 2239 {
 2240#if USE(JSVALUE64)
 2241 setupArgumentsWithExecState(arg1.gpr(), arg2.gpr());
 2242#else
 2243 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR());
 2244#endif
 2245 }
 2246
22102247 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3)
22112248 {
22122249#if USE(JSVALUE64)

@@public:
22152252 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3);
22162253#endif
22172254 }
 2255
 2256 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
 2257 {
 2258#if USE(JSVALUE64)
 2259 setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), arg3, arg4);
 2260#else
 2261 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3, arg4);
 2262#endif
 2263 }
22182264
22192265 void setupArguments(JSValueRegs arg1)
22202266 {
203461

Source/JavaScriptCore/jit/JIT.h

4343#include "Interpreter.h"
4444#include "JITDisassembler.h"
4545#include "JITInlineCacheGenerator.h"
 46#include "JITMathIC.h"
4647#include "JSInterfaceJIT.h"
4748#include "Opcode.h"
4849#include "PCToCodeOriginMap.h"
49 #include "ResultType.h"
5050#include "UnusedPointer.h"
5151
5252namespace JSC {

@@namespace JSC {
435435 void compileGetByIdHotPath(const Identifier*);
436436
437437 // Arithmetic opcode helpers
438  void emitSub32Constant(int dst, int op, int32_t constant, ResultType opType);
439438 void emitBinaryDoubleOp(OpcodeID, int dst, int op1, int op2, OperandTypes, JumpList& notInt32Op1, JumpList& notInt32Op2, bool op1IsInRegisters = true, bool op2IsInRegisters = true);
440439
441440#else // USE(JSVALUE32_64)

@@namespace JSC {
763762#endif
764763 MacroAssembler::Call callOperation(J_JITOperation_EJI, int, GPRReg, UniquedStringImpl*);
765764 MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg);
766  MacroAssembler::Call callOperation(J_JITOperation_EJJRp, JSValueRegs, JSValueRegs, JSValueRegs, ResultProfile*);
 765 MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*);
 766 MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs);
 767 MacroAssembler::Call callOperation(J_JITOperation_EJJArpJaic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, JITAddIC*);
 768 MacroAssembler::Call callOperation(J_JITOperation_EJJJaic, JSValueRegs, JSValueRegs, JSValueRegs, JITAddIC*);
767769 MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*);
768770 MacroAssembler::Call callOperation(J_JITOperation_EJJBy, int, GPRReg, GPRReg, ByValInfo*);
769771 MacroAssembler::Call callOperation(Z_JITOperation_EJOJ, GPRReg, GPRReg, GPRReg);

@@namespace JSC {
953955
954956 PCToCodeOriginMapBuilder m_pcToCodeOriginMapBuilder;
955957
 958 HashMap<Instruction*, JITAddIC*> m_instructionToJITAddIC;
 959 HashMap<Instruction*, MathICGenerationState> m_instructionToJITAddICGenerationState;
 960
956961 bool m_canBeOptimized;
957962 bool m_canBeOptimizedOrInlined;
958963 bool m_shouldEmitProfiling;
203461

Source/JavaScriptCore/jit/JITAddGenerator.cpp

2626#include "config.h"
2727#include "JITAddGenerator.h"
2828
 29#include "ArithProfile.h"
 30#include "JITMathIC.h"
 31#include "LinkBuffer.h"
 32
2933#if ENABLE(JIT)
3034
3135namespace JSC {
3236
33 void JITAddGenerator::generateFastPath(CCallHelpers& jit)
 37JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state)
 38{
 39 // We default to speculating int32.
 40 ObservedType lhs = ObservedType().withInt32();
 41 ObservedType rhs = ObservedType().withInt32();
 42 if (m_arithProfile) {
 43 lhs = m_arithProfile->lhsObservedType();
 44 rhs = m_arithProfile->rhsObservedType();
 45 if (lhs.isEmpty() || rhs.isEmpty()) {
 46 lhs = ObservedType().withInt32();
 47 rhs = ObservedType().withInt32();
 48 }
 49 }
 50
 51 if (lhs.isOnlyNonNumber() && rhs.isOnlyNonNumber())
 52 return JITMathICInlineResult::DontGenerate;
 53
 54 if ((lhs.isOnlyInt32() || m_leftOperand.isConstInt32()) && (rhs.isOnlyInt32() || m_rightOperand.isConstInt32())) {
 55 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
 56 if (!m_leftOperand.isConstInt32())
 57 state.slowPathJumps.append(jit.branchIfNotInt32(m_left));
 58 if (!m_rightOperand.isConstInt32())
 59 state.slowPathJumps.append(jit.branchIfNotInt32(m_right));
 60
 61 if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
 62 JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left;
 63 int32_t constValue = m_leftOperand.isConstInt32() ? m_leftOperand.asConstInt32() : m_rightOperand.asConstInt32();
 64 state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constValue), m_scratchGPR));
 65 } else
 66 state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
 67 jit.boxInt32(m_scratchGPR, m_result);
 68 return JITMathICInlineResult::GeneratedFastPath;
 69 }
 70
 71 return JITMathICInlineResult::GenerateFullSnippet;
 72}
 73
 74bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList)
3475{
3576 ASSERT(m_scratchGPR != InvalidGPRReg);
3677 ASSERT(m_scratchGPR != m_left.payloadGPR());

@@void JITAddGenerator::generateFastPath(C
4384
4485 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
4586
46  if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) {
47  ASSERT(!m_didEmitFastPath);
48  return;
49  }
50 
51  m_didEmitFastPath = true;
 87 if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber())
 88 return false;
5289
5390 if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
5491 JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left;

@@void JITAddGenerator::generateFastPath(C
5895 // Try to do intVar + intConstant.
5996 CCallHelpers::Jump notInt32 = jit.branchIfNotInt32(var);
6097
61  m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
 98 slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
6299
63100 jit.boxInt32(m_scratchGPR, m_result);
64  m_endJumpList.append(jit.jump());
 101 endJumpList.append(jit.jump());
65102
66103 if (!jit.supportsFloatingPoint()) {
67  m_slowPathJumpList.append(notInt32);
68  return;
 104 slowPathJumpList.append(notInt32);
 105 return true;
69106 }
70107
71108 // Try to do doubleVar + double(intConstant).
72109 notInt32.link(&jit);
73110 if (!varOpr.definitelyIsNumber())
74  m_slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
 111 slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
75112
76113 jit.unboxDoubleNonDestructive(var, m_leftFPR, m_scratchGPR, m_scratchFPR);
77114

@@void JITAddGenerator::generateFastPath(C
89126 leftNotInt = jit.branchIfNotInt32(m_left);
90127 rightNotInt = jit.branchIfNotInt32(m_right);
91128
92  m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
 129
 130 slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
93131
94132 jit.boxInt32(m_scratchGPR, m_result);
95  m_endJumpList.append(jit.jump());
 133 endJumpList.append(jit.jump());
 134
96135
97136 if (!jit.supportsFloatingPoint()) {
98  m_slowPathJumpList.append(leftNotInt);
99  m_slowPathJumpList.append(rightNotInt);
100  return;
 137 slowPathJumpList.append(leftNotInt);
 138 slowPathJumpList.append(rightNotInt);
 139 return true;
101140 }
102141
103142 leftNotInt.link(&jit);
104143 if (!m_leftOperand.definitelyIsNumber())
105  m_slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
 144 slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
106145 if (!m_rightOperand.definitelyIsNumber())
107  m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 146 slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
108147
109148 jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR);
110149 CCallHelpers::Jump rightIsDouble = jit.branchIfNotInt32(m_right);

@@void JITAddGenerator::generateFastPath(C
114153
115154 rightNotInt.link(&jit);
116155 if (!m_rightOperand.definitelyIsNumber())
117  m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
 156 slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
118157
119158 jit.convertInt32ToDouble(m_left.payloadGPR(), m_leftFPR);
120159

@@void JITAddGenerator::generateFastPath(C
128167
129168 // Do doubleVar + doubleVar.
130169 jit.addDouble(m_rightFPR, m_leftFPR);
131  if (m_resultProfile)
132  m_resultProfile->emitSetDouble(jit);
 170 if (m_arithProfile)
 171 m_arithProfile->emitSetDouble(jit);
133172
134173 jit.boxDouble(m_leftFPR, m_result);
 174
 175 return true;
135176}
136177
137178} // namespace JSC
203461

Source/JavaScriptCore/jit/JITAddGenerator.h

2929#if ENABLE(JIT)
3030
3131#include "CCallHelpers.h"
 32#include "JITMathICInlineResult.h"
 33#include "JITOperations.h"
3234#include "SnippetOperand.h"
3335
3436namespace JSC {
3537
 38struct MathICGenerationState;
 39
3640class JITAddGenerator {
3741public:
 42 JITAddGenerator()
 43 { }
 44
3845 JITAddGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
3946 JSValueRegs result, JSValueRegs left, JSValueRegs right,
4047 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
41  ResultProfile* resultProfile = nullptr)
 48 ArithProfile* arithProfile = nullptr)
4249 : m_leftOperand(leftOperand)
4350 , m_rightOperand(rightOperand)
4451 , m_result(result)

@@public:
4855 , m_rightFPR(rightFPR)
4956 , m_scratchGPR(scratchGPR)
5057 , m_scratchFPR(scratchFPR)
51  , m_resultProfile(resultProfile)
 58 , m_arithProfile(arithProfile)
5259 {
5360 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
5461 }
5562
56  void generateFastPath(CCallHelpers&);
57 
58  bool didEmitFastPath() const { return m_didEmitFastPath; }
59  CCallHelpers::JumpList& endJumpList() { return m_endJumpList; }
60  CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; }
 63 JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
 64 bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList);
6165
6266private:
6367 SnippetOperand m_leftOperand;

@@private:
6973 FPRReg m_rightFPR;
7074 GPRReg m_scratchGPR;
7175 FPRReg m_scratchFPR;
72  ResultProfile* m_resultProfile;
73  bool m_didEmitFastPath { false };
74 
75  CCallHelpers::JumpList m_endJumpList;
76  CCallHelpers::JumpList m_slowPathJumpList;
 76 ArithProfile* m_arithProfile;
7777};
7878
7979} // namespace JSC
203461

Source/JavaScriptCore/jit/JITArithmetic.cpp

2828#if ENABLE(JIT)
2929#include "JIT.h"
3030
 31#include "ArithProfile.h"
3132#include "CodeBlock.h"
3233#include "JITAddGenerator.h"
3334#include "JITBitAndGenerator.h"

3637#include "JITDivGenerator.h"
3738#include "JITInlines.h"
3839#include "JITLeftShiftGenerator.h"
 40#include "JITMathIC.h"
3941#include "JITMulGenerator.h"
4042#include "JITNegGenerator.h"
4143#include "JITOperations.h"

4547#include "JSFunction.h"
4648#include "Interpreter.h"
4749#include "JSCInlines.h"
 50#include "LinkBuffer.h"
4851#include "ResultType.h"
4952#include "SlowPathCall.h"
5053
51 
5254namespace JSC {
5355
5456void JIT::emit_op_jless(Instruction* currentInstruction)

@@void JIT::emitSlow_op_urshift(Instructio
675677 slowPathCall.call();
676678}
677679
 680ALWAYS_INLINE static OperandTypes getOperandTypes(Instruction* instruction)
 681{
 682 return OperandTypes(ArithProfile::fromInt(instruction[4].u.operand).lhsResultType(), ArithProfile::fromInt(instruction[4].u.operand).rhsResultType());
 683}
 684
678685void JIT::emit_op_add(Instruction* currentInstruction)
679686{
680687 int result = currentInstruction[1].u.operand;
681688 int op1 = currentInstruction[2].u.operand;
682689 int op2 = currentInstruction[3].u.operand;
683  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
684690
685691#if USE(JSVALUE64)
 692 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
686693 JSValueRegs leftRegs = JSValueRegs(regT0);
687694 JSValueRegs rightRegs = JSValueRegs(regT1);
688695 JSValueRegs resultRegs = leftRegs;
689696 GPRReg scratchGPR = regT2;
690697 FPRReg scratchFPR = InvalidFPRReg;
691698#else
 699 OperandTypes types = getOperandTypes(currentInstruction);
692700 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
693701 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
694702 JSValueRegs resultRegs = leftRegs;

@@void JIT::emit_op_add(Instruction* curre
696704 FPRReg scratchFPR = fpRegT2;
697705#endif
698706
699  ResultProfile* resultProfile = nullptr;
 707 ArithProfile* arithProfile = nullptr;
700708 if (shouldEmitProfiling())
701  resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
 709 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
702710
703711 SnippetOperand leftOperand(types.first());
704712 SnippetOperand rightOperand(types.second());

@@void JIT::emit_op_add(Instruction* curre
715723 if (!rightOperand.isConst())
716724 emitGetVirtualRegister(op2, rightRegs);
717725
718  JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
719  fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
 726 JITAddIC* addIC = m_codeBlock->addJITAddIC();
720727
721  gen.generateFastPath(*this);
 728 m_instructionToJITAddIC.add(currentInstruction, addIC);
 729 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
722730
723  if (gen.didEmitFastPath()) {
724  gen.endJumpList().link(this);
725  emitPutVirtualRegister(result, resultRegs);
726 
727  addSlowCase(gen.slowPathJumpList());
728  } else {
729  ASSERT(gen.endJumpList().empty());
730  ASSERT(gen.slowPathJumpList().empty());
731  if (resultProfile) {
732  if (leftOperand.isConst())
733  emitGetVirtualRegister(op1, leftRegs);
734  if (rightOperand.isConst())
735  emitGetVirtualRegister(op2, rightRegs);
736  callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
737  emitPutVirtualRegister(result, resultRegs);
738  } else {
739  JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
740  slowPathCall.call();
741  }
742  }
 731 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
 732
 733 bool generatedInlineCode = addIC->generateInline(*this, addICGenerationState);
 734 if (!generatedInlineCode) {
 735 if (leftOperand.isConst())
 736 emitGetVirtualRegister(op1, leftRegs);
 737 else if (rightOperand.isConst())
 738 emitGetVirtualRegister(op2, rightRegs);
 739
 740 if (arithProfile)
 741 callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
 742 else
 743 callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
 744 } else
 745 addSlowCase(addICGenerationState.slowPathJumps);
 746 emitPutVirtualRegister(result, resultRegs);
743747}
744748
745749void JIT::emitSlow_op_add(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
746750{
747751 linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
748752
 753 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
 754 addICGenerationState.slowPathStart = label();
 755
749756 int result = currentInstruction[1].u.operand;
750757 int op1 = currentInstruction[2].u.operand;
751758 int op2 = currentInstruction[3].u.operand;
752  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
753759
754760#if USE(JSVALUE64)
 761 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
755762 JSValueRegs leftRegs = JSValueRegs(regT0);
756763 JSValueRegs rightRegs = JSValueRegs(regT1);
757764 JSValueRegs resultRegs = leftRegs;
758765#else
 766 OperandTypes types = getOperandTypes(currentInstruction);
759767 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
760768 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
761769 JSValueRegs resultRegs = leftRegs;

@@void JIT::emitSlow_op_add(Instruction* c
769777 else if (isOperandConstantInt(op2))
770778 rightOperand.setConstInt32(getOperandConstantInt(op2));
771779
 780 if (leftOperand.isConst())
 781 emitGetVirtualRegister(op1, leftRegs);
 782 if (rightOperand.isConst())
 783 emitGetVirtualRegister(op2, rightRegs);
 784
 785 JITAddIC* addIC = m_instructionToJITAddIC.get(currentInstruction);
772786 if (shouldEmitProfiling()) {
773  if (leftOperand.isConst())
774  emitGetVirtualRegister(op1, leftRegs);
775  if (rightOperand.isConst())
776  emitGetVirtualRegister(op2, rightRegs);
777  ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
778  callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
779  emitPutVirtualRegister(result, resultRegs);
780  } else {
781  JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
782  slowPathCall.call();
783  }
 787 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
 788 if (addICGenerationState.shouldSlowPathRepatch)
 789 addICGenerationState.slowPathCall = callOperation(operationValueAddProfiledOptimize, resultRegs, leftRegs, rightRegs, &arithProfile, addIC);
 790 else
 791 addICGenerationState.slowPathCall = callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
 792 } else
 793 addICGenerationState.slowPathCall = callOperation(operationValueAddOptimize, resultRegs, leftRegs, rightRegs, addIC);
 794 emitPutVirtualRegister(result, resultRegs);
 795
 796 addLinkTask([=] (LinkBuffer& linkBuffer) {
 797 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
 798 addIC->finalizeInlineCode(addICGenerationState, linkBuffer);
 799 });
784800}
785801
786802void JIT::emit_op_div(Instruction* currentInstruction)

@@void JIT::emit_op_div(Instruction* curre
788804 int result = currentInstruction[1].u.operand;
789805 int op1 = currentInstruction[2].u.operand;
790806 int op2 = currentInstruction[3].u.operand;
791  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
792807
793808#if USE(JSVALUE64)
 809 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
794810 JSValueRegs leftRegs = JSValueRegs(regT0);
795811 JSValueRegs rightRegs = JSValueRegs(regT1);
796812 JSValueRegs resultRegs = leftRegs;
797813 GPRReg scratchGPR = regT2;
798814#else
 815 OperandTypes types = getOperandTypes(currentInstruction);
799816 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
800817 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
801818 JSValueRegs resultRegs = leftRegs;

@@void JIT::emit_op_div(Instruction* curre
803820#endif
804821 FPRReg scratchFPR = fpRegT2;
805822
806  ResultProfile* resultProfile = nullptr;
 823 ArithProfile* arithProfile = nullptr;
807824 if (shouldEmitProfiling())
808  resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
 825 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
809826
810827 SnippetOperand leftOperand(types.first());
811828 SnippetOperand rightOperand(types.second());

@@void JIT::emit_op_div(Instruction* curre
831848 emitGetVirtualRegister(op2, rightRegs);
832849
833850 JITDivGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
834  fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
 851 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
835852
836853 gen.generateFastPath(*this);
837854

@@void JIT::emit_op_mul(Instruction* curre
861878 int result = currentInstruction[1].u.operand;
862879 int op1 = currentInstruction[2].u.operand;
863880 int op2 = currentInstruction[3].u.operand;
864  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
865881
866882#if USE(JSVALUE64)
 883 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
867884 JSValueRegs leftRegs = JSValueRegs(regT0);
868885 JSValueRegs rightRegs = JSValueRegs(regT1);
869886 JSValueRegs resultRegs = JSValueRegs(regT2);
870887 GPRReg scratchGPR = regT3;
871888 FPRReg scratchFPR = InvalidFPRReg;
872889#else
 890 OperandTypes types = getOperandTypes(currentInstruction);
873891 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
874892 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
875893 JSValueRegs resultRegs = leftRegs;

@@void JIT::emit_op_mul(Instruction* curre
877895 FPRReg scratchFPR = fpRegT2;
878896#endif
879897
880  ResultProfile* resultProfile = nullptr;
 898 ArithProfile* arithProfile = nullptr;
881899 if (shouldEmitProfiling())
882  resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
 900 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
883901
884902 SnippetOperand leftOperand(types.first());
885903 SnippetOperand rightOperand(types.second());

@@void JIT::emit_op_mul(Instruction* curre
897915 emitGetVirtualRegister(op2, rightRegs);
898916
899917 JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
900  fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
 918 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
901919
902920 gen.generateFastPath(*this);
903921

@@void JIT::emit_op_mul(Instruction* curre
909927 } else {
910928 ASSERT(gen.endJumpList().empty());
911929 ASSERT(gen.slowPathJumpList().empty());
912  if (resultProfile) {
 930 if (arithProfile) {
913931 if (leftOperand.isPositiveConstInt32())
914932 emitGetVirtualRegister(op1, leftRegs);
915933 if (rightOperand.isPositiveConstInt32())
916934 emitGetVirtualRegister(op2, rightRegs);
917  callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
 935 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
918936 emitPutVirtualRegister(result, resultRegs);
919937 } else {
920938 JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);

@@void JIT::emitSlow_op_mul(Instruction* c
930948 int result = currentInstruction[1].u.operand;
931949 int op1 = currentInstruction[2].u.operand;
932950 int op2 = currentInstruction[3].u.operand;
933  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
934951
935952#if USE(JSVALUE64)
 953 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
936954 JSValueRegs leftRegs = JSValueRegs(regT0);
937955 JSValueRegs rightRegs = JSValueRegs(regT1);
938956 JSValueRegs resultRegs = leftRegs;
939957#else
 958 OperandTypes types = getOperandTypes(currentInstruction);
940959 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
941960 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
942961 JSValueRegs resultRegs = leftRegs;

@@void JIT::emitSlow_op_mul(Instruction* c
955974 emitGetVirtualRegister(op1, leftRegs);
956975 if (rightOperand.isPositiveConstInt32())
957976 emitGetVirtualRegister(op2, rightRegs);
958  ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
959  callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
 977 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
 978 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
960979 emitPutVirtualRegister(result, resultRegs);
961980 } else {
962981 JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);

@@void JIT::emit_op_sub(Instruction* curre
969988 int result = currentInstruction[1].u.operand;
970989 int op1 = currentInstruction[2].u.operand;
971990 int op2 = currentInstruction[3].u.operand;
972  OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
973991
974992#if USE(JSVALUE64)
 993 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
975994 JSValueRegs leftRegs = JSValueRegs(regT0);
976995 JSValueRegs rightRegs = JSValueRegs(regT1);
977996 JSValueRegs resultRegs = leftRegs;
978997 GPRReg scratchGPR = regT2;
979998 FPRReg scratchFPR = InvalidFPRReg;
980999#else
 1000 OperandTypes types = getOperandTypes(currentInstruction);
9811001 JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
9821002 JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
9831003 JSValueRegs resultRegs = leftRegs;

@@void JIT::emit_op_sub(Instruction* curre
9851005 FPRReg scratchFPR = fpRegT2;
9861006#endif
9871007
988  ResultProfile* resultProfile = nullptr;
 1008 ArithProfile* arithProfile = nullptr;
9891009 if (shouldEmitProfiling())
990  resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
 1010 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
9911011
9921012 SnippetOperand leftOperand(types.first());
9931013 SnippetOperand rightOperand(types.second());

@@void JIT::emit_op_sub(Instruction* curre
9961016 emitGetVirtualRegister(op2, rightRegs);
9971017
9981018 JITSubGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
999  fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
 1019 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
10001020
10011021 gen.generateFastPath(*this);
10021022

@@void JIT::emitSlow_op_sub(Instruction* c
10231043#endif
10241044
10251045 if (shouldEmitProfiling()) {
1026  ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
1027  callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
 1046 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
 1047 callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
10281048 emitPutVirtualRegister(result, resultRegs);
10291049 } else {
10301050 JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_sub);
203461

Source/JavaScriptCore/jit/JITDivGenerator.cpp

2828
2929#if ENABLE(JIT)
3030
 31#include "ArithProfile.h"
3132#include "JSCJSValueInlines.h"
3233#include "MathCommon.h"
3334

@@void JITDivGenerator::generateFastPath(C
101102 }
102103
103104 // Is the result actually an integer? The DFG JIT would really like to know. If it's
104  // not an integer, we increment a count. If this together with the slow case counter
105  // are below threshold then the DFG JIT will compile this division with a speculation
106  // that the remainder is zero.
 105 // not an integer, we set a bit. If this together with the slow case counter are below
 106 // threshold then the DFG JIT will compile this division with a speculation that the
 107 // remainder is zero.
107108
108109 // As well, there are cases where a double result here would cause an important field
109110 // in the heap to sometimes have doubles in it, resulting in double predictions getting

@@void JITDivGenerator::generateFastPath(C
128129
129130 notDoubleZero.link(&jit);
130131#endif
131  if (m_resultProfile)
132  jit.add32(CCallHelpers::TrustedImm32(1), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfSpecialFastPathCount()));
 132 if (m_arithProfile)
 133 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::specialFastPathBit), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
133134 jit.boxDouble(m_leftFPR, m_result);
134135}
135136
203461

Source/JavaScriptCore/jit/JITDivGenerator.h

@@public:
3838 JITDivGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
3939 JSValueRegs result, JSValueRegs left, JSValueRegs right,
4040 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
41  ResultProfile* resultProfile = nullptr)
 41 ArithProfile* arithProfile = nullptr)
4242 : m_leftOperand(leftOperand)
4343 , m_rightOperand(rightOperand)
4444 , m_result(result)

@@public:
4848 , m_rightFPR(rightFPR)
4949 , m_scratchGPR(scratchGPR)
5050 , m_scratchFPR(scratchFPR)
51  , m_resultProfile(resultProfile)
 51 , m_arithProfile(arithProfile)
5252 {
5353 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
5454 }

@@private:
7171 FPRReg m_rightFPR;
7272 GPRReg m_scratchGPR;
7373 FPRReg m_scratchFPR;
74  ResultProfile* m_resultProfile;
 74 ArithProfile* m_arithProfile;
7575 bool m_didEmitFastPath { false };
7676
7777 CCallHelpers::JumpList m_endJumpList;
203461

Source/JavaScriptCore/jit/JITInlines.h

@@ALWAYS_INLINE MacroAssembler::Call JIT::
416416 return appendCallWithExceptionCheck(operation);
417417}
418418
419 inline MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJRp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ResultProfile* resultProfile)
 419ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJ operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2)
420420{
421  setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(resultProfile));
 421 setupArgumentsWithExecState(arg1, arg2);
 422 Call call = appendCallWithExceptionCheck(operation);
 423 setupResults(result);
 424 return call;
 425}
 426
 427ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile)
 428{
 429 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile));
 430 Call call = appendCallWithExceptionCheck(operation);
 431 setupResults(result);
 432 return call;
 433}
 434
 435ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, JITAddIC* addIC)
 436{
 437 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), TrustedImmPtr(addIC));
 438 Call call = appendCallWithExceptionCheck(operation);
 439 setupResults(result);
 440 return call;
 441}
 442
 443ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
 444{
 445 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(addIC));
422446 Call call = appendCallWithExceptionCheck(operation);
423447 setupResults(result);
424448 return call;
203461

Source/JavaScriptCore/jit/JITMathIC.h

 1/*
 2 * Copyright (C) 2016 Apple Inc. All rights reserved.
 3 *
 4 * Redistribution and use in source and binary forms, with or without
 5 * modification, are permitted provided that the following conditions
 6 * are met:
 7 * 1. Redistributions of source code must retain the above copyright
 8 * notice, this list of conditions and the following disclaimer.
 9 * 2. Redistributions in binary form must reproduce the above copyright
 10 * notice, this list of conditions and the following disclaimer in the
 11 * documentation and/or other materials provided with the distribution.
 12 *
 13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
 14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
 17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
 21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 24 */
 25
 26#pragma once
 27
 28#if ENABLE(JIT)
 29
 30#include "CCallHelpers.h"
 31#include "JITAddGenerator.h"
 32#include "JITMathICInlineResult.h"
 33#include "LinkBuffer.h"
 34#include "Repatch.h"
 35#include "SnippetOperand.h"
 36
 37namespace JSC {
 38
 39class LinkBuffer;
 40
 41struct MathICGenerationState {
 42 MacroAssembler::Label fastPathStart;
 43 MacroAssembler::Label fastPathEnd;
 44 MacroAssembler::Label slowPathStart;
 45 MacroAssembler::Call slowPathCall;
 46 MacroAssembler::JumpList slowPathJumps;
 47 bool shouldSlowPathRepatch;
 48};
 49
 50template <typename GeneratorType>
 51class JITMathIC {
 52public:
 53 CodeLocationLabel doneLocation() { return m_inlineStart.labelAtOffset(m_inlineSize); }
 54 CodeLocationLabel slowPathStartLocation() { return m_inlineStart.labelAtOffset(m_deltaFromStartToSlowPathStart); }
 55 CodeLocationCall slowPathCallLocation() { return m_inlineStart.callAtOffset(m_deltaFromStartToSlowPathCallLocation); }
 56
 57 bool generateInline(CCallHelpers& jit, MathICGenerationState& state)
 58 {
 59 state.fastPathStart = jit.label();
 60 JITMathICInlineResult result = m_generator.generateInline(jit, state);
 61
 62 switch (result) {
 63 case JITMathICInlineResult::GeneratedFastPath: {
 64 state.shouldSlowPathRepatch = true;
 65 state.fastPathEnd = jit.label();
 66 return true;
 67 }
 68 case JITMathICInlineResult::GenerateFullSnippet: {
 69 MacroAssembler::JumpList endJumpList;
 70 bool result = m_generator.generateFastPath(jit, endJumpList, state.slowPathJumps);
 71 if (result) {
 72 state.fastPathEnd = jit.label();
 73 state.shouldSlowPathRepatch = false;
 74 endJumpList.link(&jit);
 75 return true;
 76 }
 77 return false;
 78 }
 79 case JITMathICInlineResult::DontGenerate: {
 80 return false;
 81 }
 82 default:
 83 ASSERT_NOT_REACHED();
 84 }
 85
 86 return false;
 87 }
 88
 89 void generateOutOfLine(VM& vm, CodeBlock* codeBlock, FunctionPtr callReplacement)
 90 {
 91 // We rewire to the alternate regardless of whether or not we can allocate the out of line path
 92 // because if we fail allocating the out of line path, we don't want to waist time trying to
 93 // allocate it in the future.
 94 ftlThunkAwareRepatchCall(codeBlock, slowPathCallLocation(), callReplacement);
 95
 96 {
 97 CCallHelpers jit(&vm, codeBlock);
 98
 99 MacroAssembler::JumpList endJumpList;
 100 MacroAssembler::JumpList slowPathJumpList;
 101 bool emittedFastPath = m_generator.generateFastPath(jit, endJumpList, slowPathJumpList);
 102 if (!emittedFastPath)
 103 return;
 104 endJumpList.append(jit.jump());
 105
 106 LinkBuffer linkBuffer(vm, jit, codeBlock, JITCompilationCanFail);
 107 if (linkBuffer.didFailToAllocate())
 108 return;
 109
 110 linkBuffer.link(endJumpList, doneLocation());
 111 linkBuffer.link(slowPathJumpList, slowPathStartLocation());
 112
 113 m_code = FINALIZE_CODE_FOR(
 114 codeBlock, linkBuffer, ("JITMathIC: generating out of line IC snippet"));
 115 }
 116
 117 {
 118 CCallHelpers jit(&vm, codeBlock);
 119 auto jump = jit.jump();
 120 // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
 121 bool needsBranchCompaction = false;
 122 LinkBuffer linkBuffer(jit, m_inlineStart.dataLocation(), jit.m_assembler.buffer().codeSize(), JITCompilationMustSucceed, needsBranchCompaction);
 123 RELEASE_ASSERT(linkBuffer.isValid());
 124 linkBuffer.link(jump, CodeLocationLabel(m_code.code().executableAddress()));
 125 FINALIZE_CODE(linkBuffer, ("JITMathIC: linking constant jump to out of line stub"));
 126 }
 127 }
 128
 129 void finalizeInlineCode(const MathICGenerationState& state, LinkBuffer& linkBuffer)
 130 {
 131 CodeLocationLabel start = linkBuffer.locationOf(state.fastPathStart);
 132 m_inlineStart = start;
 133
 134 m_inlineSize = MacroAssembler::differenceBetweenCodePtr(
 135 start, linkBuffer.locationOf(state.fastPathEnd));
 136 ASSERT(m_inlineSize > 0);
 137
 138 m_deltaFromStartToSlowPathCallLocation = MacroAssembler::differenceBetweenCodePtr(
 139 start, linkBuffer.locationOf(state.slowPathCall));
 140 m_deltaFromStartToSlowPathStart = MacroAssembler::differenceBetweenCodePtr(
 141 start, linkBuffer.locationOf(state.slowPathStart));
 142 }
 143
 144 MacroAssemblerCodeRef m_code;
 145 CodeLocationLabel m_inlineStart;
 146 int32_t m_inlineSize;
 147 int32_t m_deltaFromStartToSlowPathCallLocation;
 148 int32_t m_deltaFromStartToSlowPathStart;
 149 GeneratorType m_generator;
 150};
 151
 152typedef JITMathIC<JITAddGenerator> JITAddIC;
 153
 154} // namespace JSC
 155
 156#endif // ENABLE(JIT)
nonexistent

Source/JavaScriptCore/jit/JITMathICForwards.h

 1/*
 2 * Copyright (C) 2016 Apple Inc. All rights reserved.
 3 *
 4 * Redistribution and use in source and binary forms, with or without
 5 * modification, are permitted provided that the following conditions
 6 * are met:
 7 * 1. Redistributions of source code must retain the above copyright
 8 * notice, this list of conditions and the following disclaimer.
 9 * 2. Redistributions in binary form must reproduce the above copyright
 10 * notice, this list of conditions and the following disclaimer in the
 11 * documentation and/or other materials provided with the distribution.
 12 *
 13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
 14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
 17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
 21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 24 */
 25
 26#pragma once
 27
 28#if ENABLE(JIT)
 29
 30namespace JSC {
 31
 32template <typename Generator> class JITMathIC;
 33class JITAddGenerator;
 34
 35typedef JITMathIC<JITAddGenerator> JITAddIC;
 36
 37} // namespace JSC
 38
 39#endif // ENABLE(JIT)
nonexistent

Source/JavaScriptCore/jit/JITMathICInlineResult.h

 1/*
 2 * Copyright (C) 2016 Apple Inc. All rights reserved.
 3 *
 4 * Redistribution and use in source and binary forms, with or without
 5 * modification, are permitted provided that the following conditions
 6 * are met:
 7 * 1. Redistributions of source code must retain the above copyright
 8 * notice, this list of conditions and the following disclaimer.
 9 * 2. Redistributions in binary form must reproduce the above copyright
 10 * notice, this list of conditions and the following disclaimer in the
 11 * documentation and/or other materials provided with the distribution.
 12 *
 13 * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
 14 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 15 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 16 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
 17 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 18 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 19 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 20 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
 21 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 22 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 23 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 24 */
 25
 26#pragma once
 27
 28#if ENABLE(JIT)
 29
 30namespace JSC {
 31
 32enum class JITMathICInlineResult {
 33 GeneratedFastPath,
 34 GenerateFullSnippet,
 35 DontGenerate
 36};
 37
 38} // namespace JSC
 39
 40#endif // ENABLE(JIT)
nonexistent

Source/JavaScriptCore/jit/JITMulGenerator.cpp

2828
2929#if ENABLE(JIT)
3030
 31#include "ArithProfile.h"
 32
3133namespace JSC {
3234
3335void JITMulGenerator::generateFastPath(CCallHelpers& jit)

@@void JITMulGenerator::generateFastPath(C
137139 // Do doubleVar * doubleVar.
138140 jit.mulDouble(m_rightFPR, m_leftFPR);
139141
140  if (!m_resultProfile)
 142 if (!m_arithProfile)
141143 jit.boxDouble(m_leftFPR, m_result);
142144 else {
143145 // The Int52 overflow check below intentionally omits 1ll << 51 as a valid negative Int52 value.

@@void JITMulGenerator::generateFastPath(C
149151 jit.moveDoubleTo64(m_leftFPR, m_result.payloadGPR());
150152 CCallHelpers::Jump notNegativeZero = jit.branch64(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm64(negativeZeroBits));
151153
152  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 154 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
153155 CCallHelpers::Jump done = jit.jump();
154156
155157 notNegativeZero.link(&jit);
156  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 158 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
157159
158160 jit.move(m_result.payloadGPR(), m_scratchGPR);
159161 jit.urshiftPtr(CCallHelpers::Imm32(52), m_scratchGPR);
160162 jit.and32(CCallHelpers::Imm32(0x7ff), m_scratchGPR);
161163 CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
162164
163  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 165 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
164166 noInt52Overflow.link(&jit);
165167
166168 done.link(&jit);

@@void JITMulGenerator::generateFastPath(C
171173 notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm32(0)));
172174 notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.tagGPR(), CCallHelpers::TrustedImm32(negativeZeroBits >> 32)));
173175
174  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 176 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
175177 CCallHelpers::Jump done = jit.jump();
176178
177179 notNegativeZero.link(&jit);
178  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 180 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
179181
180182 jit.move(m_result.tagGPR(), m_scratchGPR);
181183 jit.urshiftPtr(CCallHelpers::Imm32(52 - 32), m_scratchGPR);
182184 jit.and32(CCallHelpers::Imm32(0x7ff), m_scratchGPR);
183185 CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
184186
185  jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
 187 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
186188
187189 m_endJumpList.append(noInt52Overflow);
188190 if (m_scratchGPR == m_result.tagGPR() || m_scratchGPR == m_result.payloadGPR())
203461

Source/JavaScriptCore/jit/JITMulGenerator.h

@@public:
3838 JITMulGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
3939 JSValueRegs result, JSValueRegs left, JSValueRegs right,
4040 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
41  ResultProfile* resultProfile = nullptr)
 41 ArithProfile* arithProfile = nullptr)
4242 : m_leftOperand(leftOperand)
4343 , m_rightOperand(rightOperand)
4444 , m_result(result)

@@public:
4848 , m_rightFPR(rightFPR)
4949 , m_scratchGPR(scratchGPR)
5050 , m_scratchFPR(scratchFPR)
51  , m_resultProfile(resultProfile)
 51 , m_arithProfile(arithProfile)
5252 {
5353 ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
5454 }

@@private:
6969 FPRReg m_rightFPR;
7070 GPRReg m_scratchGPR;
7171 FPRReg m_scratchFPR;
72  ResultProfile* m_resultProfile;
 72 ArithProfile* m_arithProfile;
7373 bool m_didEmitFastPath { false };
7474
7575 CCallHelpers::JumpList m_endJumpList;
203461

Source/JavaScriptCore/jit/JITOperations.cpp

2828
2929#if ENABLE(JIT)
3030
 31#include "ArithProfile.h"
3132#include "ArrayConstructor.h"
3233#include "CommonSlowPaths.h"
3334#include "DFGCompilationMode.h"

@@JSCell* JIT_OPERATION operationToIndexSt
22472248 return jsString(exec, Identifier::from(exec, index).string());
22482249}
22492250
 2251ALWAYS_INLINE static EncodedJSValue unprofiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
 2252{
 2253 VM* vm = &exec->vm();
 2254 NativeCallFrameTracer tracer(vm, exec);
 2255
 2256 JSValue op1 = JSValue::decode(encodedOp1);
 2257 JSValue op2 = JSValue::decode(encodedOp2);
 2258
 2259 return JSValue::encode(jsAdd(exec, op1, op2));
 2260}
 2261
 2262ALWAYS_INLINE static EncodedJSValue profiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
 2263{
 2264 VM* vm = &exec->vm();
 2265 NativeCallFrameTracer tracer(vm, exec);
 2266
 2267 JSValue op1 = JSValue::decode(encodedOp1);
 2268 JSValue op2 = JSValue::decode(encodedOp2);
 2269
 2270 ASSERT(arithProfile);
 2271 arithProfile->observeLHSAndRHS(op1, op2);
 2272
 2273 JSValue result = jsAdd(exec, op1, op2);
 2274 arithProfile->detectNumericness(result);
 2275
 2276 return JSValue::encode(result);
 2277}
 2278
22502279EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
22512280{
 2281 return unprofiledAdd(exec, encodedOp1, encodedOp2);
 2282}
 2283
 2284EncodedJSValue JIT_OPERATION operationValueAddNotProfiledNotOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*)
 2285{
 2286 return unprofiledAdd(exec, encodedOp1, encodedOp2);
 2287}
 2288
 2289EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
 2290{
 2291 return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
 2292}
 2293
 2294EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC* addIC)
 2295{
22522296 VM* vm = &exec->vm();
22532297 NativeCallFrameTracer tracer(vm, exec);
22542298
22552299 JSValue op1 = JSValue::decode(encodedOp1);
22562300 JSValue op2 = JSValue::decode(encodedOp2);
 2301
 2302 ASSERT(arithProfile);
 2303 arithProfile->observeLHSAndRHS(op1, op2);
 2304 auto nonOptimizeVariant = operationValueAddProfiledNonOptimize;
 2305 addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
22572306
 2307 JSValue result = jsAdd(exec, op1, op2);
 2308 arithProfile->detectNumericness(result);
 2309
 2310 return JSValue::encode(result);
 2311}
 2312
 2313EncodedJSValue JIT_OPERATION operationValueAddProfiledNonOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC*)
 2314{
 2315 return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
 2316}
 2317
 2318EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC* addIC)
 2319{
 2320 VM* vm = &exec->vm();
 2321 NativeCallFrameTracer tracer(vm, exec);
 2322
 2323 auto nonOptimizeVariant = operationValueAddNonOptimize;
 2324 addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
 2325
 2326 JSValue op1 = JSValue::decode(encodedOp1);
 2327 JSValue op2 = JSValue::decode(encodedOp2);
22582328 return JSValue::encode(jsAdd(exec, op1, op2));
22592329}
22602330
2261 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
 2331EncodedJSValue JIT_OPERATION operationValueAddNonOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*)
22622332{
22632333 VM* vm = &exec->vm();
22642334 NativeCallFrameTracer tracer(vm, exec);

@@EncodedJSValue JIT_OPERATION operationVa
22672337 JSValue op2 = JSValue::decode(encodedOp2);
22682338
22692339 JSValue result = jsAdd(exec, op1, op2);
2270  resultProfile->detectNumericness(result);
 2340
22712341 return JSValue::encode(result);
22722342}
22732343

@@EncodedJSValue JIT_OPERATION operationVa
22842354 return JSValue::encode(jsNumber(a * b));
22852355}
22862356
2287 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
 2357EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
22882358{
22892359 VM* vm = &exec->vm();
22902360 NativeCallFrameTracer tracer(vm, exec);

@@EncodedJSValue JIT_OPERATION operationVa
22962366 double b = op2.toNumber(exec);
22972367
22982368 JSValue result = jsNumber(a * b);
2299  resultProfile->detectNumericness(result);
 2369 arithProfile->detectNumericness(result);
23002370 return JSValue::encode(result);
23012371}
23022372

@@EncodedJSValue JIT_OPERATION operationVa
23132383 return JSValue::encode(jsNumber(a - b));
23142384}
23152385
2316 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
 2386EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
23172387{
23182388 VM* vm = &exec->vm();
23192389 NativeCallFrameTracer tracer(vm, exec);

@@EncodedJSValue JIT_OPERATION operationVa
23252395 double b = op2.toNumber(exec);
23262396
23272397 JSValue result = jsNumber(a - b);
2328  resultProfile->detectNumericness(result);
 2398 arithProfile->detectNumericness(result);
23292399 return JSValue::encode(result);
23302400}
23312401
203461

Source/JavaScriptCore/jit/JITOperations.h

2828
2929#if ENABLE(JIT)
3030
 31#include "JITMathICForwards.h"
3132#include "MacroAssemblerCodeRef.h"
3233#include "PropertyOffset.h"
3334#include "SlowPathReturnType.h"

@@class ArrayProfile;
4243class CallLinkInfo;
4344class CodeBlock;
4445class ExecState;
 46class JITAddGenerator;
4547class JSArray;
4648class JSFunction;
4749class JSLexicalEnvironment;

@@class WatchpointSet;
5456
5557struct ByValInfo;
5658struct InlineCallFrame;
57 struct ResultProfile;
 59struct ArithProfile;
5860
5961typedef ExecState CallFrame;
6062

@@typedef char* UnusedPtr;
6971 A: JSArray*
7072 Aap: ArrayAllocationProfile*
7173 Ap: ArrayProfile*
 74 Arp: ArithProfile*
7275 By: ByValInfo*
7376 C: JSCell*
7477 Cb: CodeBlock*

@@typedef char* UnusedPtr;
8184 Icf: InlineCallFrame*
8285 Idc: const Identifier*
8386 J: EncodedJSValue
 87 Jaic: JITAddIC*
8488 Jcp: const JSValue*
8589 Jsc: JSScope*
8690 Jsf: JSFunction*

@@typedef char* UnusedPtr;
9296 Q: int64_t
9397 R: Register
9498 Reo: RegExpObject*
95  Rp: ResultProfile*
9699 S: size_t
97100 Sprt: SlowPathReturnType
98101 Ssi: StructureStubInfo*

@@typedef EncodedJSValue (JIT_OPERATION *J
133136typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJAp)(ExecState*, EncodedJSValue, EncodedJSValue, ArrayProfile*);
134137typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJBy)(ExecState*, EncodedJSValue, EncodedJSValue, ByValInfo*);
135138typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue);
136 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJRp)(ExecState*, EncodedJSValue, EncodedJSValue, ResultProfile*);
 139typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArp)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*);
 140typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpJaic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, JITAddIC*);
 141typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJaic)(ExecState*, EncodedJSValue, EncodedJSValue, JITAddIC*);
137142typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t);
138143typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*);
139144typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReoJss)(ExecState*, JSString*, RegExpObject*, JSString*);

@@EncodedJSValue JIT_OPERATION operationNe
408413JSCell* JIT_OPERATION operationToIndexString(ExecState*, int32_t);
409414
410415EncodedJSValue JIT_OPERATION operationValueAdd(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
411 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
 416EncodedJSValue JIT_OPERATION operationValueAddNotProfiledNotOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
 417EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
 418EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
 419EncodedJSValue JIT_OPERATION operationValueAddProfiledNonOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
 420EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
 421EncodedJSValue JIT_OPERATION operationValueAddNonOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
412422EncodedJSValue JIT_OPERATION operationValueMul(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
413 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
 423EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
414424EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
415 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
 425EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
416426
417427void JIT_OPERATION operationProcessTypeProfilerLog(ExecState*) WTF_INTERNAL;
418428void JIT_OPERATION operationProcessShadowChickenLog(ExecState*) WTF_INTERNAL;
203461

Source/JavaScriptCore/jit/JITSubGenerator.cpp

2626#include "config.h"
2727#include "JITSubGenerator.h"
2828
 29#include "ArithProfile.h"
 30
2931#if ENABLE(JIT)
3032
3133namespace JSC {

@@void JITSubGenerator::generateFastPath(C
8385 rightWasInteger.link(&jit);
8486
8587 jit.subDouble(m_rightFPR, m_leftFPR);
86  if (m_resultProfile)
87  m_resultProfile->emitSetDouble(jit);
 88 if (m_arithProfile)
 89 m_arithProfile->emitSetDouble(jit);
8890
8991 jit.boxDouble(m_leftFPR, m_result);
9092}
203461

Source/JavaScriptCore/jit/JITSubGenerator.h

@@public:
3838 JITSubGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
3939 JSValueRegs result, JSValueRegs left, JSValueRegs right,
4040 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
41  ResultProfile* resultProfile = nullptr)
 41 ArithProfile* arithProfile = nullptr)
4242 : m_leftOperand(leftOperand)
4343 , m_rightOperand(rightOperand)
4444 , m_result(result)

@@public:
4848 , m_rightFPR(rightFPR)
4949 , m_scratchGPR(scratchGPR)
5050 , m_scratchFPR(scratchFPR)
51  , m_resultProfile(resultProfile)
 51 , m_arithProfile(arithProfile)
5252 { }
5353
5454 void generateFastPath(CCallHelpers&);

@@private:
6767 FPRReg m_rightFPR;
6868 GPRReg m_scratchGPR;
6969 FPRReg m_scratchFPR;
70  ResultProfile* m_resultProfile;
 70 ArithProfile* m_arithProfile;
7171 bool m_didEmitFastPath { false };
7272
7373 CCallHelpers::JumpList m_endJumpList;
203461

Source/JavaScriptCore/jit/Repatch.cpp

@@static FunctionPtr readCallTarget(CodeBl
7272 return result;
7373}
7474
75 static void repatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
 75void ftlThunkAwareRepatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
7676{
7777#if ENABLE(FTL_JIT)
7878 if (codeBlock->jitType() == JITCode::FTLJIT) {

@@static InlineCacheAction tryCacheGetByID
167167
168168 bool generatedCodeInline = InlineAccess::generateArrayLength(*codeBlock->vm(), stubInfo, jsCast<JSArray*>(baseValue));
169169 if (generatedCodeInline) {
170  repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
 170 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
171171 stubInfo.initArrayLength();
172172 return RetryCacheLater;
173173 }

@@static InlineCacheAction tryCacheGetByID
220220 if (generatedCodeInline) {
221221 LOG_IC((ICEvent::GetByIdSelfPatch, structure->classInfo(), propertyName));
222222 structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset());
223  repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
 223 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
224224 stubInfo.initGetByIdSelf(codeBlock, structure, slot.cachedOffset());
225225 return RetryCacheLater;
226226 }

@@void repatchGetByID(ExecState* exec, JSV
314314 GCSafeConcurrentJITLocker locker(exec->codeBlock()->m_lock, exec->vm().heap);
315315
316316 if (tryCacheGetByID(exec, baseValue, propertyName, slot, stubInfo, kind) == GiveUpOnCache)
317  repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
 317 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
318318}
319319
320320static V_JITOperation_ESsiJJI appropriateGenericPutByIdFunction(const PutPropertySlot &slot, PutKind putKind)

@@static InlineCacheAction tryCachePutByID
372372 bool generatedCodeInline = InlineAccess::generateSelfPropertyReplace(vm, stubInfo, structure, slot.cachedOffset());
373373 if (generatedCodeInline) {
374374 LOG_IC((ICEvent::PutByIdSelfPatch, structure->classInfo(), ident));
375  repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
 375 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
376376 stubInfo.initPutByIdReplace(codeBlock, structure, slot.cachedOffset());
377377 return RetryCacheLater;
378378 }

@@void repatchPutByID(ExecState* exec, JSV
468468 GCSafeConcurrentJITLocker locker(exec->codeBlock()->m_lock, exec->vm().heap);
469469
470470 if (tryCachePutByID(exec, baseValue, structure, propertyName, slot, stubInfo, putKind) == GiveUpOnCache)
471  repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
 471 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
472472}
473473
474474static InlineCacheAction tryRepatchIn(

@@void repatchIn(
529529{
530530 SuperSamplerScope superSamplerScope(false);
531531 if (tryRepatchIn(exec, base, ident, wasFound, slot, stubInfo) == GiveUpOnCache)
532  repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
 532 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
533533}
534534
535535static void linkSlowFor(VM*, CallLinkInfo& callLinkInfo, MacroAssemblerCodeRef codeRef)

@@void linkPolymorphicCall(
901901
902902void resetGetByID(CodeBlock* codeBlock, StructureStubInfo& stubInfo, GetByIDKind kind)
903903{
904  repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
 904 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
905905 InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
906906}
907907

@@void resetPutByID(CodeBlock* codeBlock,
920920 optimizedFunction = operationPutByIdDirectNonStrictOptimize;
921921 }
922922
923  repatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
 923 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
924924 InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
925925}
926926
203461

Source/JavaScriptCore/jit/Repatch.h

@@void linkPolymorphicCall(ExecState*, Cal
5454void resetGetByID(CodeBlock*, StructureStubInfo&, GetByIDKind);
5555void resetPutByID(CodeBlock*, StructureStubInfo&);
5656void resetIn(CodeBlock*, StructureStubInfo&);
 57void ftlThunkAwareRepatchCall(CodeBlock*, CodeLocationCall, FunctionPtr newCalleeFunction);
5758
5859} // namespace JSC
5960
203461

Source/JavaScriptCore/llint/LLIntData.cpp

2626#include "config.h"
2727#include "LLIntData.h"
2828
 29#include "ArithProfile.h"
2930#include "BytecodeConventions.h"
3031#include "CodeBlock.h"
3132#include "CodeType.h"

@@void Data::performAssertions(VM& vm)
223224#endif
224225
225226 ASSERT(StringImpl::s_hashFlag8BitBuffer == 8);
 227
 228 {
 229 uint32_t bits = 0x120000;
 230 UNUSED_PARAM(bits);
 231 ArithProfile arithProfile;
 232 arithProfile.lhsSawInt32();
 233 arithProfile.rhsSawInt32();
 234 ASSERT(arithProfile.bits() == bits);
 235 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
 236 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
 237 }
 238 {
 239 uint32_t bits = 0x220000;
 240 UNUSED_PARAM(bits);
 241 ArithProfile arithProfile;
 242 arithProfile.lhsSawNumber();
 243 arithProfile.rhsSawInt32();
 244 ASSERT(arithProfile.bits() == bits);
 245 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
 246 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
 247 }
 248 {
 249 uint32_t bits = 0x240000;
 250 UNUSED_PARAM(bits);
 251 ArithProfile arithProfile;
 252 arithProfile.lhsSawNumber();
 253 arithProfile.rhsSawNumber();
 254 ASSERT(arithProfile.bits() == bits);
 255 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
 256 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
 257 }
 258 {
 259 uint32_t bits = 0x140000;
 260 UNUSED_PARAM(bits);
 261 ArithProfile arithProfile;
 262 arithProfile.lhsSawInt32();
 263 arithProfile.rhsSawNumber();
 264 ASSERT(arithProfile.bits() == bits);
 265 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
 266 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
 267 }
226268}
227269#if COMPILER(CLANG)
228270#pragma clang diagnostic pop
203461

Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm

@@macro binaryOpCustomStore(integerOperati
10141014 loadConstantOrVariable2Reg(t0, t2, t0)
10151015 bineq t2, Int32Tag, .op1NotInt
10161016 bineq t3, Int32Tag, .op2NotInt
 1017 loadisFromInstruction(4, t5)
 1018 ori ArithProfileIntInt, t5
 1019 storeisToInstruction(t5, 4)
10171020 loadi 4[PC], t2
10181021 integerOperationAndStore(t3, t1, t0, .slow, t2)
10191022 dispatch(5)

@@macro binaryOpCustomStore(integerOperati
10231026 bia t2, LowestTag, .slow
10241027 bib t3, LowestTag, .op1NotIntOp2Double
10251028 bineq t3, Int32Tag, .slow
 1029 loadisFromInstruction(4, t5)
 1030 ori ArithProfileNumberInt, t5
 1031 storeisToInstruction(t5, 4)
10261032 ci2d t1, ft1
10271033 jmp .op1NotIntReady
10281034.op1NotIntOp2Double:
10291035 fii2d t1, t3, ft1
 1036 loadisFromInstruction(4, t5)
 1037 ori ArithProfileNumberNumber, t5
 1038 storeisToInstruction(t5, 4)
10301039.op1NotIntReady:
10311040 loadi 4[PC], t1
10321041 fii2d t0, t2, ft0

@@macro binaryOpCustomStore(integerOperati
10381047 # First operand is definitely an int, the second operand is definitely not.
10391048 loadi 4[PC], t2
10401049 bia t3, LowestTag, .slow
 1050 loadisFromInstruction(4, t5)
 1051 ori ArithProfileIntNumber, t5
 1052 storeisToInstruction(t5, 4)
10411053 ci2d t0, ft0
10421054 fii2d t1, t3, ft1
10431055 doubleOperation(ft1, ft0)
203461

Source/JavaScriptCore/llint/LowLevelInterpreter64.asm

@@macro binaryOpCustomStore(integerOperati
893893 bqb t1, tagTypeNumber, .op2NotInt
894894 loadisFromInstruction(1, t2)
895895 integerOperationAndStore(t1, t0, .slow, t2)
 896 loadisFromInstruction(4, t1)
 897 ori ArithProfileIntInt, t1
 898 storeisToInstruction(t1, 4)
896899 dispatch(5)
897900
898901.op1NotInt:

@@macro binaryOpCustomStore(integerOperati
902905 btqz t1, tagTypeNumber, .slow
903906 addq tagTypeNumber, t1
904907 fq2d t1, ft1
 908 loadisFromInstruction(4, t2)
 909 ori ArithProfileNumberNumber, t2
 910 storeisToInstruction(t2, 4)
905911 jmp .op1NotIntReady
906912.op1NotIntOp2Int:
 913 loadisFromInstruction(4, t2)
 914 ori ArithProfileNumberInt, t2
 915 storeisToInstruction(t2, 4)
907916 ci2d t1, ft1
908917.op1NotIntReady:
909918 loadisFromInstruction(1, t2)

@@macro binaryOpCustomStore(integerOperati
919928 # First operand is definitely an int, the second is definitely not.
920929 loadisFromInstruction(1, t2)
921930 btqz t1, tagTypeNumber, .slow
 931 loadisFromInstruction(4, t3)
 932 ori ArithProfileIntNumber, t3
 933 storeisToInstruction(t3, 4)
922934 ci2d t0, ft0
923935 addq tagTypeNumber, t1
924936 fq2d t1, ft1
203461

Source/JavaScriptCore/llint/LowLevelInterpreter.asm

@@const IsInvalidated = 2
252252# ShadowChicken data
253253const ShadowChickenTailMarker = 0x7a11
254254
 255# ArithProfile data
 256const ArithProfileIntInt = 0x120000
 257const ArithProfileNumberInt = 0x220000
 258const ArithProfileNumberNumber = 0x240000
 259const ArithProfileIntNumber = 0x140000
 260
255261# Some register conventions.
256262if JSVALUE64
257263 # - Use a pair of registers to represent the PC: one register for the

@@if JSVALUE64
286292 loadp offset * 8[PB, PC, 8], dest
287293 end
288294
 295 macro storeisToInstruction(value, offset)
 296 storei value, offset * 8[PB, PC, 8]
 297 end
 298
289299 macro storepToInstruction(value, offset)
290300 storep value, offset * 8[PB, PC, 8]
291301 end

@@else
299309 macro loadpFromInstruction(offset, dest)
300310 loadp offset * 4[PC], dest
301311 end
 312
 313 macro storeisToInstruction(value, offset)
 314 storei value, offset * 4[PC]
 315 end
302316end
303317
304318if X86_64_WIN
203461

Source/JavaScriptCore/parser/ResultType.h

@@namespace JSC {
3232 private:
3333 friend struct OperandTypes;
3434
35  typedef char Type;
 35 typedef uint8_t Type;
3636 static const Type TypeInt32 = 1;
37 
38  static const Type TypeMaybeNumber = 0x04;
39  static const Type TypeMaybeString = 0x08;
40  static const Type TypeMaybeNull = 0x10;
41  static const Type TypeMaybeBool = 0x20;
42  static const Type TypeMaybeOther = 0x40;
 37 static const Type TypeMaybeNumber = 0x02;
 38 static const Type TypeMaybeString = 0x04;
 39 static const Type TypeMaybeNull = 0x08;
 40 static const Type TypeMaybeBool = 0x10;
 41 static const Type TypeMaybeOther = 0x20;
4342
4443 static const Type TypeBits = TypeMaybeNumber | TypeMaybeString | TypeMaybeNull | TypeMaybeBool | TypeMaybeOther;
4544
 45 public:
 46 static const int numBitsNeeded = 6;
 47 static_assert((TypeBits & ((1 << numBitsNeeded) - 1)) == TypeBits, "This is necessary for correctness.");
 48
4649 explicit ResultType(Type type)
47  : m_type(type)
 50 : m_bits(type)
4851 {
4952 }
5053
51  public:
5254 bool isInt32() const
5355 {
54  return m_type & TypeInt32;
 56 return m_bits & TypeInt32;
5557 }
5658
5759 bool definitelyIsNumber() const
5860 {
59  return (m_type & TypeBits) == TypeMaybeNumber;
 61 return (m_bits & TypeBits) == TypeMaybeNumber;
6062 }
6163
6264 bool definitelyIsString() const
6365 {
64  return (m_type & TypeBits) == TypeMaybeString;
 66 return (m_bits & TypeBits) == TypeMaybeString;
6567 }
6668
6769 bool definitelyIsBoolean() const
6870 {
69  return (m_type & TypeBits) == TypeMaybeBool;
 71 return (m_bits & TypeBits) == TypeMaybeBool;
7072 }
7173
7274 bool mightBeNumber() const
7375 {
74  return m_type & TypeMaybeNumber;
 76 return m_bits & TypeMaybeNumber;
7577 }
7678
7779 bool isNotNumber() const

@@namespace JSC {
141143 return numberTypeIsInt32();
142144 }
143145
 146 Type bits() const { return m_bits; }
 147
144148 private:
145  Type m_type;
 149 Type m_bits;
146150 };
147151
148152 struct OperandTypes

@@namespace JSC {
152156 // We have to initialize one of the int to ensure that
153157 // the entire struct is initialized.
154158 m_u.i = 0;
155  m_u.rds.first = first.m_type;
156  m_u.rds.second = second.m_type;
 159 m_u.rds.first = first.m_bits;
 160 m_u.rds.second = second.m_bits;
157161 }
158162
159163 union {
203461

Source/JavaScriptCore/runtime/CommonSlowPaths.cpp

2626#include "config.h"
2727#include "CommonSlowPaths.h"
2828
 29#include "ArithProfile.h"
2930#include "ArrayConstructor.h"
3031#include "BuiltinNames.h"
3132#include "CallFrame.h"

@@SLOW_PATH_DECL(slow_path_negate)
359360}
360361
361362#if ENABLE(DFG_JIT)
362 static void updateResultProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
 363static void updateArithProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
363364{
364365 CodeBlock* codeBlock = exec->codeBlock();
365  unsigned bytecodeOffset = codeBlock->bytecodeOffset(pc);
366  ResultProfile* profile = codeBlock->ensureResultProfile(bytecodeOffset);
 366 ArithProfile& profile = codeBlock->arithProfileForPC(pc);
367367
368368 if (result.isNumber()) {
369369 if (!result.isInt32()) {
370370 if (left.isInt32() && right.isInt32())
371  profile->setObservedInt32Overflow();
 371 profile.setObservedInt32Overflow();
372372
373373 double doubleVal = result.asNumber();
374374 if (!doubleVal && std::signbit(doubleVal))
375  profile->setObservedNegZeroDouble();
 375 profile.setObservedNegZeroDouble();
376376 else {
377  profile->setObservedNonNegZeroDouble();
 377 profile.setObservedNonNegZeroDouble();
378378
379379 // The Int52 overflow check here intentionally omits 1ll << 51 as a valid negative Int52 value.
380380 // Therefore, we will get a false positive if the result is that value. This is intentionally

@@static void updateResultProfileForBinary
382382 static const int64_t int52OverflowPoint = (1ll << 51);
383383 int64_t int64Val = static_cast<int64_t>(std::abs(doubleVal));
384384 if (int64Val >= int52OverflowPoint)
385  profile->setObservedInt52Overflow();
 385 profile.setObservedInt52Overflow();
386386 }
387387 }
388388 } else
389  profile->setObservedNonNumber();
 389 profile.setObservedNonNumber();
390390}
391391#else
392 static void updateResultProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
 392static void updateArithProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
393393#endif
394394
395395SLOW_PATH_DECL(slow_path_to_number)

@@SLOW_PATH_DECL(slow_path_add)
407407 JSValue v2 = OP_C(3).jsValue();
408408 JSValue result;
409409
 410 ArithProfile& arithProfile = exec->codeBlock()->arithProfileForPC(pc);
 411 arithProfile.observeLHSAndRHS(v1, v2);
 412
410413 if (v1.isString() && !v2.isObject())
411414 result = jsString(exec, asString(v1), v2.toString(exec));
412415 else if (v1.isNumber() && v2.isNumber())

@@SLOW_PATH_DECL(slow_path_add)
415418 result = jsAddSlowCase(exec, v1, v2);
416419
417420 RETURN_WITH_PROFILING(result, {
418  updateResultProfileForBinaryArithOp(exec, pc, result, v1, v2);
 421 updateArithProfileForBinaryArithOp(exec, pc, result, v1, v2);
419422 });
420423}
421424

@@SLOW_PATH_DECL(slow_path_mul)
432435 double b = right.toNumber(exec);
433436 JSValue result = jsNumber(a * b);
434437 RETURN_WITH_PROFILING(result, {
435  updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
 438 updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
436439 });
437440}
438441

@@SLOW_PATH_DECL(slow_path_sub)
445448 double b = right.toNumber(exec);
446449 JSValue result = jsNumber(a - b);
447450 RETURN_WITH_PROFILING(result, {
448  updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
 451 updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
449452 });
450453}
451454

@@SLOW_PATH_DECL(slow_path_div)
458461 double b = right.toNumber(exec);
459462 JSValue result = jsNumber(a / b);
460463 RETURN_WITH_PROFILING(result, {
461  updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
 464 updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
462465 });
463466}
464467
203461

Source/JavaScriptCore/tests/stress/op-add-exceptions.js

 1function assert(b) {
 2 if (!b)
 3 throw new Error("Bad!");
 4}
 5noInline(assert);
 6
 7function f1() { return "f1"; }
 8noInline(f1);
 9function f2() { return "f2"; }
 10noInline(f2);
 11function f3() { return "f3"; }
 12noInline(f3);
 13
 14let oException = {
 15 valueOf() { throw new Error(""); }
 16};
 17
 18function foo(arg1, arg2) {
 19 let a = f1();
 20 let b = f2();
 21 let c = f3();
 22 try {
 23 arg1 + arg2;
 24 } catch(e) {
 25 assert(arg1 === oException);
 26 assert(arg2 === oException);
 27 }
 28 assert(a === "f1");
 29 assert(b === "f2");
 30 assert(c === "f3");
 31}
 32noInline(foo);
 33
 34for (let i = 0; i < 1000; i++) {
 35 foo(i, {});
 36 foo({}, i);
 37}
 38foo(oException, oException);
 39for (let i = 0; i < 10000; i++) {
 40 foo(i, {});
 41 foo({}, i);
 42}
 43foo(oException, oException);
 44
 45
 46function ident(x) { return x; }
 47noInline(ident);
 48
 49function bar(arg1, arg2) {
 50 let a = f1();
 51 let b = f2();
 52 let c = f3();
 53 let x = ident(arg1);
 54 let y = ident(arg2);
 55
 56 try {
 57 arg1 + arg2;
 58 } catch(e) {
 59 assert(arg1 === oException);
 60 assert(arg2 === oException);
 61 assert(x === oException);
 62 assert(y === oException);
 63 }
 64 assert(a === "f1");
 65 assert(b === "f2");
 66 assert(c === "f3");
 67}
 68noInline(bar);
 69
 70for (let i = 0; i < 1000; i++) {
 71 bar(i, {});
 72 bar({}, i);
 73}
 74bar(oException, oException);
 75for (let i = 0; i < 10000; i++) {
 76 bar(i, {});
 77 bar({}, i);
 78}
 79bar(oException, oException);
nonexistent

Source/WebCore/ChangeLog

 12016-07-20 Saam Barati <sbarati@apple.com>
 2
 3 op_add/ValueAdd should be an IC in all JIT tiers
 4 https://bugs.webkit.org/show_bug.cgi?id=159649
 5
 6 Reviewed by NOBODY (OOPS!).
 7
 8 * ForwardingHeaders/jit/JITMathICForwards.h: Added.
 9
1102016-07-20 Chris Dumez <cdumez@apple.com>
211
312 CSSStyleDeclaration.setProperty() should be able to unset "important" on a property
203461

Source/WebCore/ForwardingHeaders/jit/JITMathICForwards.h

 1#ifndef WebCore_FWD_JITMathICForwards_h
 2#define WebCore_FWD_JITMathICForwards_h
 3#include <JavaScriptCore/JITMathICForwards.h>
 4#endif
nonexistent